14.06.2013 Views

6th European Conference on Information Management - Academic ...

6th European Conference on Information Management - Academic ...

6th European Conference on Information Management - Academic ...

SHOW MORE
SHOW LESS

Create successful ePaper yourself

Turn your PDF publications into a flip-book with our unique Google optimized e-Paper software.

Proceedings<br />

of the<br />

<str<strong>on</strong>g>6th</str<strong>on</strong>g> <str<strong>on</strong>g>European</str<strong>on</strong>g> <str<strong>on</strong>g>C<strong>on</strong>ference</str<strong>on</strong>g> <strong>on</strong><br />

Informati<strong>on</strong> <strong>Management</strong><br />

and Evaluati<strong>on</strong><br />

University College Cork<br />

Ireland<br />

13-14 September 2012<br />

Edited by<br />

Dr Tadgh Nagle<br />

University College Cork<br />

Ireland


Copyright The Authors, 2012. All Rights Reserved.<br />

No reproducti<strong>on</strong>, copy or transmissi<strong>on</strong> may be made without written permissi<strong>on</strong> from the individual authors.<br />

Papers have been double-blind peer reviewed before final submissi<strong>on</strong> to the c<strong>on</strong>ference. Initially, paper abstracts<br />

were read and selected by the c<strong>on</strong>ference panel for submissi<strong>on</strong> as possible papers for the c<strong>on</strong>ference.<br />

Many thanks to the reviewers who helped ensure the quality of the full papers.<br />

These <str<strong>on</strong>g>C<strong>on</strong>ference</str<strong>on</strong>g> Proceedings have been submitted to Thoms<strong>on</strong> ISI for indexing.<br />

Further copies of this book and previous year’s proceedings can be purchased from<br />

http://academic-bookshop.com<br />

CD versi<strong>on</strong> ISBN: 978-1-908272-66-9<br />

CD versi<strong>on</strong> ISSN: 2048-979X<br />

Book versi<strong>on</strong> ISBN: 978-1-908272-65-2<br />

Book Versi<strong>on</strong> ISSN: 2048-8912<br />

Published by <strong>Academic</strong> Publishing Internati<strong>on</strong>al Limited<br />

Reading<br />

UK<br />

44-118-972-4148<br />

www.academic-publishing.org


C<strong>on</strong>tents<br />

Paper Title Author(s) Page<br />

Preface iv<br />

Committee v<br />

Biographies vii<br />

Proposal of Adaptability Indexes to<br />

Support <strong>Management</strong> of Engineering and Marketing<br />

Systems<br />

Knowledge Gaps in Post-Merger Integrati<strong>on</strong> of Software<br />

Maintenance Processes: A Case Study<br />

A Proposed Framework for Guiding the Effective<br />

Implementati<strong>on</strong> of an Informal Communicati<strong>on</strong> System<br />

for Virtual Teams<br />

i<br />

No.<br />

Oswaldo Luiz Agostinho 1<br />

Maria Alaranta and Eero Martela 9<br />

Garth Alistoun and Christopher Upfold 17<br />

IS C<strong>on</strong>sultants and SMEs: A Competence Perspective Adrian Bradshaw, Paul Cragg and Venkat<br />

Pulakanam<br />

Developing a Framework for Maturing IT Risk<br />

<strong>Management</strong> Capabilities<br />

IS Evaluati<strong>on</strong> in the Fusi<strong>on</strong> View: An Emergence<br />

Perspective<br />

Where do Tablets fit in the Organizati<strong>on</strong>’s Workstati<strong>on</strong><br />

Inventory?<br />

Classifying IT Investment Evaluati<strong>on</strong> Methods According<br />

to Functi<strong>on</strong>al Criteri<strong>on</strong><br />

<strong>Academic</strong> Group and Forum <strong>on</strong> Facebook: Social, Serious<br />

Studies or Synergy?<br />

Evaluating the Process of Delivering Compelling Value<br />

Propositi<strong>on</strong>s: The Case of Mobile Payments<br />

Using Bricolage to Facilitate Emergent Collectives in<br />

SMEs<br />

Determining the Maturity Level of eCommerce in South<br />

African SMEs<br />

Advancing GeoMarketing Analyses with Improved<br />

Spatio-temporal Distributi<strong>on</strong> of Populati<strong>on</strong> at High<br />

Resoluti<strong>on</strong><br />

Activity Theory: A Useful Evaluati<strong>on</strong> Methodology for the<br />

Role of Informati<strong>on</strong> Systems in Collaborative Activity<br />

Dealing With Uncertainty Through KM: Cases in Four<br />

Software SMEs<br />

Analyzing Less<strong>on</strong>s Learned to Identify Potential Risks in<br />

new Product Development Projects<br />

Evaluating Determinants for ERP use and Value in<br />

Scandinavia: Exploring Differences Between Danish and<br />

Swedish SMEs<br />

Marian Carcary 33<br />

Sven Carlss<strong>on</strong> and Olgerta T<strong>on</strong>a 41<br />

Mitch Cochran and Paul Witman 47<br />

Jacek Cypryjanski<br />

Ruth de Villiers and Marco Cobus Pretorius 63<br />

Denis Dennehy, Frederic Adam and Fergal<br />

Cart<strong>on</strong><br />

Jan Devos, Hendrik Van Landeghem and<br />

Dirk Deschoolmeester<br />

David Freeme and Portia Gumede 91<br />

Sérgio Freire and Teresa Santos 100<br />

Audrey Grace 109<br />

Ciara Heavin and Frederic Adam 117<br />

Vered Holzmann 127<br />

Björn Johanss<strong>on</strong>, Pedro Ruivo, Tiago<br />

Oliveira and Miguel Neto<br />

25<br />

55<br />

74<br />

82<br />

135


Paper Title Author(s) Page<br />

User Experience in Mobile Ph<strong>on</strong>es by Using Semantic<br />

Differential Methodology<br />

Challenges in Building a Community Health Informati<strong>on</strong><br />

Exchange in a Complex Envir<strong>on</strong>ment<br />

Factors Inhibiting Recogniti<strong>on</strong> and Reporting of Losses<br />

From Cyber-Attacks: The Case of Government<br />

Departments in the Western Cape Province of South<br />

Africa<br />

The Overall Process Taken by Enterprises to Manage the<br />

IaaS Cloud Services<br />

Sustainable Enterprise Architecture: A Three-<br />

Dimensi<strong>on</strong>al Framework for <strong>Management</strong> of<br />

Architectural Change<br />

Applying Structural Equati<strong>on</strong> Modelling to Exploring the<br />

Relati<strong>on</strong>ship Between Organisati<strong>on</strong>al Trust and Quality<br />

of Work Life<br />

Identificati<strong>on</strong> and Governance of Emerging Ethical Issues<br />

in Informati<strong>on</strong> Systems: Empirical and Theoretical<br />

Presuppositi<strong>on</strong>s<br />

Breaking C<strong>on</strong>sensus in IS Evaluati<strong>on</strong>s: The Agitati<strong>on</strong><br />

Workshop<br />

Drivers and Challenges for Biometrics in the Financial<br />

Services<br />

ii<br />

No.<br />

Kalimullah Khan 143<br />

Ranjan Kini 151<br />

Michael Kyobe, Sinka Matengu , Proske<br />

Walter and Mzwandile Sh<strong>on</strong>gwe<br />

Alina Mădălina L<strong>on</strong>ea, Daniela Elena<br />

Popescu and Octavian Proştean<br />

Thanos Magoulas, Aida Hadzic, Ted<br />

Saarikko and Kalevi Pessi<br />

159<br />

168<br />

178<br />

Nico Martins and Yolandi van der Berg 186<br />

Laurence Masclet and Philippe Gouj<strong>on</strong> 195<br />

John McAvoy, Tadhg Nagle and David<br />

Samm<strong>on</strong><br />

203<br />

Karen Neville et al 211<br />

Did you get Your Facebook Sessi<strong>on</strong> Completed? Markku Nurminen 219<br />

Infusi<strong>on</strong> of Mobile Health Systems in the NHS: An<br />

Empirical Study<br />

An Exploratory Study of Innovati<strong>on</strong> Intermediati<strong>on</strong> in IS<br />

Educati<strong>on</strong><br />

Bringing Some Order to the ‘Black Art’ of Innovati<strong>on</strong><br />

Measurement<br />

Using Focus Groups to Evaluate Artefacts in Design<br />

Research<br />

Realizing the Business Value of Service-Oriented<br />

Architecture: The C<strong>on</strong>structi<strong>on</strong> of a Theoretical<br />

Framework<br />

The Identificati<strong>on</strong> of Service Oriented Architecture-<br />

Specific Critical Success Factors<br />

Treasure Hunting in the 21st century: A Decade of<br />

Geocaching in Portugal<br />

Intelligent Decisi<strong>on</strong> Support Systems Development Based<br />

<strong>on</strong> Modern Modeling Methods<br />

Yv<strong>on</strong>ne O’ C<strong>on</strong>nor, John O’ D<strong>on</strong>oghue and<br />

Phillip O’ Reilly<br />

226<br />

Brian O’Flaherty and Joe Bogue 234<br />

Paidi O’Raghallaigh, David Samm<strong>on</strong> and<br />

Ciaran Murphy<br />

Paidi O’Raghallaigh, David Samm<strong>on</strong> and<br />

Ciaran Murphy<br />

R<strong>on</strong>an O’Sullivan, Tom Butler and Philip<br />

O’Reilly<br />

Ian Owens and John Cunningham<br />

Teresa Santos, Ricardo Mendes, António<br />

Rodrigues and Sérgio Freire<br />

243<br />

251<br />

258<br />

267<br />

273<br />

Elena Serova 282<br />

Integrating Sustainability Indicators in IT/IS Evaluati<strong>on</strong> Gilbert Silvius 291<br />

The art of Shooting the Moving Goal – Explorative Study<br />

of EA Pilot<br />

Nestori Syynimaa 302<br />

Informati<strong>on</strong> Interacti<strong>on</strong> in Terms of eCommerce Kamila Tislerova 307


Paper Title Author(s) Page<br />

Designing High Quality ICT for Altered Envir<strong>on</strong>mental<br />

C<strong>on</strong>diti<strong>on</strong>s<br />

An Analysis of the Problems Linked to Ec<strong>on</strong>omic<br />

Evaluati<strong>on</strong> of <strong>Management</strong> Support Informati<strong>on</strong> Systems<br />

in Poland <strong>on</strong> the Example of ERP/CRM Class Applicati<strong>on</strong>s<br />

- Problem Analysis<br />

Towards an Understanding of Enterprise Architecture<br />

Analysis Activities<br />

Moving Towards a Sensor-Based Patient M<strong>on</strong>itoring<br />

System: Evaluating its Impact <strong>on</strong> Data and Informati<strong>on</strong><br />

Quality<br />

Using the REA Approach to Modeling of IT Process<br />

Evaluati<strong>on</strong><br />

iii<br />

Daryoush Daniel Vaziri, Dirk Schreiber and<br />

Andreas Gadatsch<br />

No.<br />

314<br />

Bartosz Wachnik 326<br />

Haining Wan and Sven Carlss<strong>on</strong> 334<br />

Atieh Zarabzadeh, John O’D<strong>on</strong>oghue,<br />

Frederic Adam, Mervyn O’C<strong>on</strong>nell, Siobhán<br />

O’C<strong>on</strong>nor, Sim<strong>on</strong> Woodworth, Joe<br />

Gallagher and Tom O’Kane<br />

342<br />

Ryszard Zygala 351<br />

N<strong>on</strong> <strong>Academic</strong> papers 361<br />

A Process Model to Guarantee Informati<strong>on</strong> Quality in<br />

Elective Surgery Informati<strong>on</strong> Systems<br />

Rita Cristóvão and Pedro Gomes 363<br />

PHD papers 373<br />

Informati<strong>on</strong> Risks and Their C<strong>on</strong>necti<strong>on</strong> With Accounting Marie Černá 375<br />

A Methodology for Competitive Intelligence Metrics Rhiann<strong>on</strong> Gainor and France Bouthillier 383<br />

The use of Virtual Public Space and eCommunities to<br />

Kick-Start eParticipati<strong>on</strong> – Timisoara, Romania<br />

Strategic <strong>Management</strong> and Informati<strong>on</strong> Evaluati<strong>on</strong><br />

Challenges Facing Entrepreneurs of SMEs in ICT<br />

Method Engineering Approach to the Adopti<strong>on</strong> of<br />

Informati<strong>on</strong> Technology Governance, Risk and<br />

Compliance in Swiss Hospitals<br />

Collaborative Methodology for Supply Chain Quality<br />

<strong>Management</strong>: Framework and Integrati<strong>on</strong> With Strategic<br />

Decisi<strong>on</strong> Processes in Product Development<br />

M<strong>on</strong>ica Izvercianu and Ana-Maria Branea 391<br />

Maroun Jneid and Antoine Tannous 400<br />

Mike Krey, Steven Furnell, Bettina<br />

Harriehausen and Matthias Knoll<br />

Juan Camilo Romero, Thierry Coudert, Laurent<br />

Geneste and Aymeric De Valroger<br />

Work In Progress papers 429<br />

Recording our Professi<strong>on</strong>al Development Just Became<br />

Easier: Using a Learning <strong>Management</strong> System<br />

Is More Data Better? Experiences From Measuring<br />

<strong>Academic</strong> Performance<br />

408<br />

418/<br />

Mercy Kesiena Clement-Okooboh 431<br />

Harald Lothaller 435


Preface<br />

The <str<strong>on</strong>g>6th</str<strong>on</strong>g> <str<strong>on</strong>g>European</str<strong>on</strong>g> <str<strong>on</strong>g>C<strong>on</strong>ference</str<strong>on</strong>g> <strong>on</strong> Informati<strong>on</strong> <strong>Management</strong> and Evaluati<strong>on</strong> (ECIME) is hosted this year by University<br />

College Cork in Ireland. The <str<strong>on</strong>g>C<strong>on</strong>ference</str<strong>on</strong>g> Chair is Dr David Samm<strong>on</strong> and the Programme Chair is Dr Tadhg Nagle, both<br />

from University College Cork.<br />

ECIME provides an opportunity for individuals researching and working in the broad field of informati<strong>on</strong> management,<br />

including informati<strong>on</strong> technology evaluati<strong>on</strong> to come together to exchange ideas and discuss current research in the<br />

field. We hope that this year’s c<strong>on</strong>ference will provide you with plenty of opportunities to share your expertise with<br />

colleagues from around the world.<br />

The opening keynote address will be delivered by Professor Patrick Finnegan, University of New South Wales, Sydney,<br />

Australia <strong>on</strong> the topic ProSIS: An explorati<strong>on</strong> of potential Pro-Social and Pro-Societal impact through Informati<strong>on</strong> Systems.<br />

ECIME 2012 received an initial submissi<strong>on</strong> of 125 abstracts. After the double-blind peer review process 43 academic<br />

papers, 1 n<strong>on</strong>-academic paper, 6 PhD papers and 2 short work in progress papers have been accepted for these <str<strong>on</strong>g>C<strong>on</strong>ference</str<strong>on</strong>g><br />

Proceedings. These papers represent research from around the world, including Austria, Belgium, Brazil, Canada,<br />

Czech Republic, Finland, Germany, Ireland, Israel, Kingdom of Saudi Arabia, Leban<strong>on</strong>, Netherlands, New Zealand,<br />

Poland, Portugal, Romania, Russia, South Africa, Sweden, Switzerland, UK, USA.<br />

We wish you a most interesting c<strong>on</strong>ference.<br />

Dr TadhgNagle<br />

Programme Chair<br />

Dr David Samm<strong>on</strong><br />

<str<strong>on</strong>g>C<strong>on</strong>ference</str<strong>on</strong>g> Chair<br />

September 2012<br />

iv


<str<strong>on</strong>g>C<strong>on</strong>ference</str<strong>on</strong>g> Executive<br />

Dr Frank Bannister Trinity College, Dublin<br />

Professor Eg<strong>on</strong> Berghout, Gr<strong>on</strong>ingen University, Netherlands<br />

Dr Ann Brown, City University Business School, L<strong>on</strong>d<strong>on</strong><br />

Dr Walter Castelnovo, University of Insubria, Como, Italy<br />

Dr Elena Ferrari, University of Insubria, Como, Italy<br />

Mini Track chairs<br />

Dr Maria Alaranta, Aalto University, Finland<br />

Dr Jorge Ferreira, Nova University, Lisb<strong>on</strong>, Portugal<br />

Dr Ciara Heavin, University College Cork, Ireland<br />

Dr Karen Neville, University College Cork, Ireland<br />

Le<strong>on</strong>a O’Brien, University College Cork, Ireland<br />

Ian Owens, Cranfield University, UK<br />

Isa Santos, University of Porto, Portugal<br />

Dr Elena Serova, St. Petersburg State University of Ec<strong>on</strong>omics and Finance, Russia<br />

Dr João Manuel R. S. Tavares, University of Porto, Portugal<br />

<str<strong>on</strong>g>C<strong>on</strong>ference</str<strong>on</strong>g> Committee<br />

The c<strong>on</strong>ference programme committee c<strong>on</strong>sists of key people in the informati<strong>on</strong> systems community. The following<br />

people have c<strong>on</strong>firmed their participati<strong>on</strong>:<br />

Ademola Adesina (University of Western Cape, South Africa); Maria Alaranta (Helsinki University of Technology TKK,<br />

Finland); Saheer Al-Jaghoub (Al-Ahliyya Amman University, Jordan); Maria Ceu Alves (University of Beira Interior, Portugal);<br />

Hussein Al-Yaseen (Amman University, Jordan); Karen Anders<strong>on</strong> (Mid Sweden University, Sweden); Joan Ballantine<br />

(University of Ulster, UK); Frank Bannister (Trinity College Dublin, Ireland); Ofer Barkai (SCE - Sami Shamo<strong>on</strong><br />

College of Engineering, Israel); David Barnes (Westminster Business School, University of Westminster, L<strong>on</strong>d<strong>on</strong>, UK);<br />

Peter Bednar (Department of ISCA, Portsmouth University, UK); Eg<strong>on</strong> Berghout (University of Gr<strong>on</strong>ingen, The Netherlands);<br />

Milena Bobeva (Bournemouth University, Poole, UK); Ann Brown (CASS Business School, L<strong>on</strong>d<strong>on</strong>, UK); Giovanni<br />

Camp<strong>on</strong>ovo (University of Applied Sciences of Southern Switzerland, Switzerland); Sven Carlss<strong>on</strong> (School of Ec<strong>on</strong>omics<br />

and <strong>Management</strong>, Lund University, Sweden); Fergal Cart<strong>on</strong> (University College Cork, Ireland); Walter Castelnovo (Università<br />

dell’Insubria, Como, Italy); Anna Cavallo (University of Rome, "Sapienza", Italy); Sunil Choenni (University of<br />

Twente and Ministry of Justice, The Netherlands); Peter Clutterbuck (University of Queensland, Australia); Reet Cr<strong>on</strong>k<br />

(Harding University, Texas, USA); Barbara Crump (Massey University, New Zealand); Renata Dameri (University of<br />

Genoa, Italy); Paul Davies (University of Glamorgan, UK); Miguel de Castro Neto ( ISEGI, Universidade Nova de Lisboa,,<br />

Portugal); Guillermo de Haro (Instituto de Empresa, Madrid, Spain); Francois Deltour (GET-ENST-Bretagne Engineering<br />

School, France); Dirk Deschoolmeester (Ghent University, Belgium); Jan Devos (Ghent University, Belgium,); Eduardo<br />

Diniz (Escola de Administracao de Empresas de Sao Paulo, Fundacao Getulio Vargas, Brazil); Maria do Rosário Martins<br />

(Universidade Cape Verde, Portugal); Romano Dyers<strong>on</strong> (Royal Holloway University, L<strong>on</strong>d<strong>on</strong>, UK); Alea Fairchild (Vesalius<br />

College/Vrije Univ Brussels, Belgium); Jorge Ferreira (e-Geo Geography and Regi<strong>on</strong>al Planning Research Centre /<br />

New University of Lisb<strong>on</strong>, Portugal); Graham Fletcher (Cranfield University / Defence Academy of the UK, UK); Elisabeth<br />

Frisk (Chalmers University of Technology, Göteborg, Sweden); Andreas Gadatsch (B<strong>on</strong>n-Rhein-Sieg University of<br />

Applied Sciences, Germany); Ken Grant (Ryers<strong>on</strong> University, Tor<strong>on</strong>to, Canada); Ginevra Gravili (Facolta Di Ec<strong>on</strong>omia,<br />

Lecce, Italy); Paul Griffiths (The Birchman Group, Santiago, Chile); Kerstin Grundén (Trollhattan University, Sweden);<br />

Loshma Gunisetti (Sri Vasavi Engineering College, India); Petri Hallikainen (University of Sydney, Business School, Australia);<br />

Ciara Heavin (University College Cork, Ireland); J<strong>on</strong>as Hedman (Copenhagen Business School, Denmark); Matthew<br />

Hint<strong>on</strong> (Open University Business School, UK); Vered Holzmann (Tel-Aviv University / Hol<strong>on</strong> Institute of Technology,<br />

Israel); Grant Royd Howard (University of South Africa (UNISA), South Africa); Björn Johanss<strong>on</strong> (Lund University,<br />

Sweden); Paul J<strong>on</strong>es (University of Glamorgan, UK); Ghassan Kbar (Riyadh Techno Valley, King Saud University, Saudi<br />

Arabia); Ranjan Kini (Indiana University Northwest, Gary, USA); Lutz Kirchner (BOC Informati<strong>on</strong> Technologies C<strong>on</strong>sulting<br />

GmbH Voßstr. 22, Germany); Juha K<strong>on</strong>tio (Turku University of Applied Sciences, Finland); Jussi Koskinen (University<br />

of Jyvaskyla, Finland); Luigi Lavazza (Università degli Studi dell'Insubria, Italy); Przemysław Lech (University of Gdańsk,<br />

Poland); Sam Lubbe (University of South Africa, South Africa); Paolo Magrassi (Polytechnique of Milan, Italy); P<strong>on</strong>nusamy<br />

Manohar (University of Papua New Guinea, Papua New Guinea); Nenad Markovic (Belgrade Business School,<br />

Serbia); Steve Martin (University of East L<strong>on</strong>d<strong>on</strong>, UK); Milos Maryska (University of Ec<strong>on</strong>omics, Prague, Czech Republic);<br />

John McAvoy (University College Cork, Ireland); Nor Laila Md Noor (Universiti Teknologi MARA, Malaysia); Annette<br />

Mills (University of Canterbury, Christchurch, New Zealand); Maria Mitre (Universidad de Oviedo, Spain); Mahmoud<br />

Moradi (University of Guilan, Rasht, Iran); Gunilla Myreteg (Uppsala University, Sweden); Mário Negas (Aberta Univer-<br />

v


sity, Portugal); Karen Neville (University College Cork, Ireland); Emil Numminen (Blekinge Institute of Technology,<br />

Sweden); Tiago Oliveira (Universidade Nova de Lisboa, Portugal); Roslina Othman (Internati<strong>on</strong>al Islamic University<br />

Malaysia, Kuala Lumpur, Malaysia); Ian Owens (Cranfield University, Shrivenham, UK); Sevgi Özkan (Middle East Technical<br />

University, Ankara, Turkey); Shaun Pather (Cape Peninsula University of Technology, South Africa); Kalevi Pessi (IT<br />

University, Gothenburg, Sweden); Danilo Piaggesi (F<strong>on</strong>dazi<strong>on</strong>e Rosselli Americas, USA); Elias Pimenidis (University of<br />

East L<strong>on</strong>d<strong>on</strong>, UK); Zijad Pita (RMIT University, Melbourne, Australia); Nayem Rahman (Intel Corporati<strong>on</strong>, Aloha, USA);<br />

Hugo Rehesaar (NSW, Sydney, Australia); João Manuel Ribeiro da Silva Tavares (Faculdade de Engenharia da Universidade<br />

do Porto, Portugal); Dimitris Rigas (De M<strong>on</strong>tfort University, UK); Narcyz Roztocki (State University of New York at<br />

New Paltz, USA); Hannu Salmela (Turku School of Ec<strong>on</strong>omics and Business Administrati<strong>on</strong>, Finland); David Samm<strong>on</strong><br />

(University College Cork, Ireland); Elsje Scott (University of Cape Town, R<strong>on</strong>debosch, South Africa); Elena Serova<br />

(Graduate School of <strong>Management</strong> St. Petersburg State University, Russia); Yilun Shang (University of Texas at San Ant<strong>on</strong>io,<br />

USA); Hossein Sharif (University of Portsmouth, UK); A.J. Gilbert Silvius (Utrecht University of Professi<strong>on</strong>al Educati<strong>on</strong>,<br />

The Netherlands); Riccardo Spinelli (Universita Di Genova, Italy); Darijus Strasunskas (Norwegian University of<br />

Science and Technology, Tr<strong>on</strong>dheim, Norway); Reima Suomi (University of Turku, Finland); Lars Svenss<strong>on</strong> (University<br />

West, Trollhättan, Sweden); Jarmo Tähkäpää (Turku School of Ec<strong>on</strong>omics and Business Administrati<strong>on</strong>, Finland); Torben<br />

Tambo (Aarhus University, Denmark); Llewellyn Tang (University of Reading, UK); Claudine Toffol<strong>on</strong> (Université du<br />

Mans - IUT de Laval, France); Geert-Jan Van Bussel (HvA University of Applied Sciences, Amsterdam, The Netherlands);<br />

Minh<strong>on</strong>g Wang (The University of H<strong>on</strong>g K<strong>on</strong>g, H<strong>on</strong>g K<strong>on</strong>g); Anna Wingkvist (School of Computer Science, Physics and<br />

Mathematics, Linnaeus University, Sweden); Les Worrall (University of Coventry, UK); Tuan Yu (Kent Business School,<br />

University of Kent, Canterbury, UK).<br />

vi


Biographies<br />

<str<strong>on</strong>g>C<strong>on</strong>ference</str<strong>on</strong>g> Chair<br />

Programme Chair<br />

Dr. David Samm<strong>on</strong> is a researcher/lecturer in Business Informati<strong>on</strong> Systems at University College<br />

Cork. His current research interests focus <strong>on</strong> the areas of c<strong>on</strong>ceptual data modeling,<br />

data/informati<strong>on</strong> management, theory and theory-building, and redesigning organisati<strong>on</strong>al routines<br />

through mindfulness. David has published extensively in internati<strong>on</strong>al journals and c<strong>on</strong>ferences.<br />

He is an Associate Editor of the ‘Journal of Decisi<strong>on</strong> Systems’ and co-author of the book ‘Enterprise<br />

Resource Planning Era: Less<strong>on</strong>s Learned and Issues for the Future’ (2004).<br />

Dr. Tadgh Nagle is a lecturer in Business Informati<strong>on</strong> Systems at University College Cork. Coming<br />

from an industry background in finance services he became a Business Analyst Lab Leader in the<br />

Digital Enterprise Research Institute (DERI). During this time his main focus was <strong>on</strong> researching<br />

innovati<strong>on</strong> and the impact of emerging technologies <strong>on</strong> the Irish eLearning sector. From this he<br />

has c<strong>on</strong>tinued his research in strategic innovati<strong>on</strong> and the impact of disruptive technologies. Primarily<br />

exploring c<strong>on</strong>cepts such as ambidexterity, business models and emerging technologies<br />

(Web 2.0) he has published in numerous internati<strong>on</strong>al journals and c<strong>on</strong>ferences.<br />

Keynote Speaker<br />

Professor Patrick Finnegan is Professor of Informati<strong>on</strong> Systems and Head of the School of Informati<strong>on</strong><br />

Systems, Technology and <strong>Management</strong> at the University of New South Wales, Sydney. He is a<br />

Senior Editor of the Informati<strong>on</strong> Systems Journal and a Past-President of the Irish Chapter of the<br />

Associati<strong>on</strong> for Informati<strong>on</strong> Systems. He was awarded the 2011 Stafford Beer Medal with Philip<br />

O'Reilly for their work <strong>on</strong> developing a theory of electr<strong>on</strong>ic marketplace performance. His research<br />

<strong>on</strong> inter-organisati<strong>on</strong>al systems, e-business and open strategies has been published in the proceedings<br />

of leading IS c<strong>on</strong>ferences and in a variety of journals (including Informati<strong>on</strong> Systems Research, the <str<strong>on</strong>g>European</str<strong>on</strong>g><br />

Journal of Informati<strong>on</strong> Systems, the Informati<strong>on</strong> Systems Journal, the Journal of Informati<strong>on</strong> Technology, the Journal<br />

of Strategic Informati<strong>on</strong> Systems, Informati<strong>on</strong> Technology and People, the Internati<strong>on</strong>al Journal of Electr<strong>on</strong>ic Commerce,<br />

DATABASE, and Electr<strong>on</strong>ic Markets).<br />

Mini Track Chairs<br />

of IS.<br />

Dr Maria Alaranta,<br />

D.Sc. (Ec<strong>on</strong>. & Bus. Adm.) is a Visiting Research Scholar at CEPRIN, Georgia<br />

State University, USA, and a Senior Researcher (<strong>on</strong> research leave) in the Department of Industrial<br />

Engineering and <strong>Management</strong>, Aalto University, Finland. She has published a number of articles in<br />

reputable internati<strong>on</strong>al peer-refereed journals and c<strong>on</strong>ferences, including Informati<strong>on</strong> Systems<br />

Fr<strong>on</strong>tiers, the Hawaii Internati<strong>on</strong>al <str<strong>on</strong>g>C<strong>on</strong>ference</str<strong>on</strong>g> <strong>on</strong> System Sciences and the Academy of <strong>Management</strong><br />

<str<strong>on</strong>g>C<strong>on</strong>ference</str<strong>on</strong>g> etc. Dr. Alaranta has also carried out several large c<strong>on</strong>sulting projects in the area<br />

Dr Jorge Ferreira is an assistant professor at the Geography and Regi<strong>on</strong>al Planning Department,<br />

Faculty of Social Sciences and Humanities (FCSH), Nova University of Lisb<strong>on</strong>. He develops its main<br />

research activities in e-Geo, Research Centre for Geography and Regi<strong>on</strong>al Planning, an R&D unit<br />

with Government annual funding within the University. His main research interests focus <strong>on</strong> "Geography<br />

of Knowledge Society", “Geographical Informati<strong>on</strong> Technologies" and “Informati<strong>on</strong> Diffusi<strong>on</strong>”.<br />

Regi<strong>on</strong>al ec<strong>on</strong>omy and innovati<strong>on</strong> are also transversal research areas crossed by his research.<br />

Dr Ciara Heavin is a College Lecturer in Business Informati<strong>on</strong> Systems at University College Cork,<br />

Ireland. She also holds a BSc and MSc in Informati<strong>on</strong> Systems from UCC. Her main research interests<br />

include the development of the ICT industry, primarily focusing <strong>on</strong> Ireland’s software industry<br />

and knowledge management in software SMEs.<br />

vii


Dr Karen Neville is a researcher and lecturer in Business Informati<strong>on</strong> Systems (BIS) at University<br />

College Cork (UCC), Ireland. Her current research interests focus <strong>on</strong> the areas of ISS and Compliance,<br />

Social Learning and Biometrics. Karen has published in internati<strong>on</strong>al c<strong>on</strong>ferences and journals<br />

Le<strong>on</strong>a O’Brien is a graduate of UCC completing a BCL and an LLM with h<strong>on</strong>ours. She also holds a<br />

BBus from Cork Institute of Technology. Her research interests are Financial Services Law and<br />

Policy, specifically banking law, corporate governance and regulatory frameworks. She has completed<br />

research for the C<strong>on</strong>sumer Panel of the Central Bank of Ireland, Bank of Ireland, and ITPS<br />

am<strong>on</strong>gst others.<br />

Ian Owens is a lecturer and researcher at Cranfield University. His research interests include informati<strong>on</strong><br />

systems evaluati<strong>on</strong>, informati<strong>on</strong> systems development methodologies, sense making and<br />

mindfulness, enterprise architecture, and service oriented architecture. He is also researching Enterprise<br />

Architecture tools and techniques for the Defence Science and Technology Labs (DSTL<br />

Isa Santos has a BSc degree (5 years) in Mechanical Engineering and a MSc degree in Industrial<br />

Design by University of Porto. Currently, she is pursuing a PhD from the MIT | Portugal program.<br />

Her main area of interest is product development of medical devices.<br />

Dr Elena G. Serova is an Associate Professor in the Internati<strong>on</strong>al School of Ec<strong>on</strong>omics and Politics<br />

and also in the Informatics Department of St Petersburg State University of Ec<strong>on</strong>omics and<br />

Finance, Russia. Her research interests include Business Models and Modelling; Informati<strong>on</strong><br />

<strong>Management</strong> and Informati<strong>on</strong> Systems and Ec<strong>on</strong>omics of Innovati<strong>on</strong> and Project <strong>Management</strong>.<br />

Dr João Manuel R. S. Tavares graduated in Mechanical Engineering from the University of Porto –<br />

Portugalin 1992. He obtained MSc and the PhD degrees in Electr<strong>on</strong>ic and Computer Engineering, in<br />

1995 and 2001, respectively, also from the same University. Since 2001, he has been Assistant Professor<br />

in the Department of Mechanical Engineering of the Faculty of Engineering of the University<br />

of Porto, and senior researcher and project coordinator at the Institute of Mechanical<br />

Engineeringand Industrial <strong>Management</strong>. His main research areas include Medical Imaging, Biomechanics,<br />

Biomedical Engineering and New Product Development<br />

Biographies of Presenting Authors<br />

Ana-Maria Branea is a PhD student at the “Politehnica” University of Timisoara, in the field of urban management,<br />

with a background in architecture and urbanism and 6 year experience at the Research Centre for Urban Planning<br />

Timisoara, Faculty of Architecture.<br />

Marian Carcary is a post-doctoral researcher working <strong>on</strong> an IT Capability Maturity Framework research project at the<br />

Innovati<strong>on</strong> Value Institute, Nati<strong>on</strong>al University of Ireland, Maynooth. Marian previously worked as a member of Faculty<br />

in the University of Limerick and Limerick Institute of Technology. She has an MSc by research and a PhD in IT evaluati<strong>on</strong>.<br />

Sven Carlss<strong>on</strong> is Professor of Informatics at Lund University School of Ec<strong>on</strong>omics and <strong>Management</strong>. His current research<br />

interests include: Business Intelligence, KM, and enterprise 2.0. He has published more than 125 peer-reviewed<br />

journal articles, book chapters, and c<strong>on</strong>ference papers. His work has appeared in journals like JMIS, Decisi<strong>on</strong> Sciences,<br />

and Informati<strong>on</strong> Systems Journal.<br />

Dr Fergal Cart<strong>on</strong> is a College Lecturer at University College Cork. Fergal's research domain is the integrati<strong>on</strong> of informati<strong>on</strong><br />

technologies into management decisi<strong>on</strong> making. With 15 years' experience as a management c<strong>on</strong>sultant, Fer-<br />

viii


gal has a primary degree in Computer Science (University College Dublin), and an MSc in <strong>Management</strong> from the <str<strong>on</strong>g>European</str<strong>on</strong>g><br />

School of <strong>Management</strong> Studies (ESCP-EAP) in Paris.<br />

Mercy Kesiena Clement-Okooboh is a Doctoral researcher at the University of Bolt<strong>on</strong>, United Kingdom. Her research<br />

is focused <strong>on</strong> the effectiveness of different types of learning in the organisati<strong>on</strong> and her research interests include<br />

program and technology evaluati<strong>on</strong>, acti<strong>on</strong> research methodology and inquiry-based learning. Currently, she is the<br />

Head of Learning of Development in Veolia Energy (Dalkia) Ireland.<br />

Mitch Cochran has been the Informati<strong>on</strong> Systems Manager for the City of M<strong>on</strong>rovia for 14 years. He has also worked<br />

in the Court System and for IBM. He is currently working <strong>on</strong> his Informati<strong>on</strong> Systems PhD from Clarem<strong>on</strong>t Graduate<br />

University and has completed Masters Degrees in Administrati<strong>on</strong> and Homeland Security. He has a CISM certificati<strong>on</strong>.<br />

Paul Cragg is Professor of Informati<strong>on</strong> Systems at the University of Canterbury, New Zealand. He received his PhD<br />

from Loughborough University, England. He teaches the management of IS across a range of courses from undergraduate<br />

through to PhD. His research is focused <strong>on</strong> IS in SMEs, and has published in numerous journals.<br />

Fred Creed<strong>on</strong> is a PhD candidate in the Department of Business Informati<strong>on</strong> Systems at University College Cork. He<br />

received his B.Bs in BIS, and M.Bs in BIS. His primary areas of research include the introducti<strong>on</strong> of IS based early warning<br />

systems in a clinical envir<strong>on</strong>ment and their impact <strong>on</strong> decisi<strong>on</strong> making processes within the envir<strong>on</strong>ment.<br />

Rita Cristóvão, raised in Lisb<strong>on</strong>, Portugal, graduated in 2001 from Nova School of Business and Ec<strong>on</strong>omics and postgraduated<br />

in 2004 from INDEG Business School. In 2001 worked as management c<strong>on</strong>sultant at Deloitte in the<br />

healthcare sector. In 2004 started to work in SIGIC and nowadays is the assistant coordinator of SIGIC in the Central<br />

Administrati<strong>on</strong> of NHS.<br />

Jacek Cypryjanski, Ph.D. is a associated professor in the Department of IT in <strong>Management</strong>, Faculty of Ec<strong>on</strong>omics and<br />

<strong>Management</strong> at the University of Szczecin, Poland. His current research interest includes ec<strong>on</strong>omic evaluati<strong>on</strong> of informati<strong>on</strong><br />

technology investments and multiple-criteria decisi<strong>on</strong> analysis methods.<br />

Aymeric de Valroger has a MSc. degree in Industrial Engineering from the Georgia Institute of Technology in the US<br />

and a BSc. degree from the Ecole Centrale de Paris in France. Mr. A.d V. is an experienced process supply chain c<strong>on</strong>sultant<br />

and project manager. He has a proven track record of successful Supply Chain process improvement projects<br />

the last 10 years. He is director from a c<strong>on</strong>sulting firm <strong>on</strong> Supply Chain <strong>Management</strong> and Industrial Systems.<br />

Ruth de Villiers is a research professor in the School of Computing at the University of South Africa. She has a PhD,<br />

and masters’ degrees in Informati<strong>on</strong> Systems and Computer-Integrated Educati<strong>on</strong>. Her research areas are Human-<br />

Computer Interacti<strong>on</strong> and e-Learning. She c<strong>on</strong>ducts research and supervises postgraduate students in research, development,<br />

and usability evaluati<strong>on</strong> of e-learning envir<strong>on</strong>ments.<br />

Denis Dennehy is currently undertaking a PhD with the Business Informati<strong>on</strong> Systems department in UCC. His research<br />

explores the process of creating and sharing value in a mobile payment ecosystem by leveraging the business model<br />

c<strong>on</strong>cept and its associated processes. Prior to this he completed a research masters which was motivated by his work<br />

in a developing country and he also holds a BSc. in Business Informati<strong>on</strong> Systems.<br />

Jan Devos is professor in Informati<strong>on</strong> Systems at Ghent University, faculty of Engineering and Architecture, campus<br />

West. Devos has a PhD in Engineering, Industrial <strong>Management</strong>. His current research interests are IT governance in<br />

SME’s, Cloud computing, E-Business and IT security. He has published several articles <strong>on</strong> IT and SMEs and was a<br />

speaker at internati<strong>on</strong>al academic and business c<strong>on</strong>ferences.<br />

Cathal Doyle is currently pursuing a PhD in Business Informati<strong>on</strong> Systems at University College of Cork (UCC), Ireland.<br />

His research focuses <strong>on</strong> the emerging phenomena of social media and the established area of learning, where he is<br />

attempting to develop the learning envir<strong>on</strong>ments of 2020.<br />

David Freeme is a lecturer at Rhodes Universiry, Grahamstown , South Africa. He lectures IS Theory, eBusiness Strategy,<br />

Accounting Informati<strong>on</strong> Systems, and IS <strong>Management</strong> to graduate and under graduates.<br />

Sérgio Freire is a research assistant at e-GEO Research Centre for Geography and Regi<strong>on</strong>al Planning, Universidade<br />

Nova de Lisboa, Portugal. With a master’s degree in geography from the University of Kansas (USA), he has worked at<br />

ix


the Nati<strong>on</strong>al Center for Geographic Informati<strong>on</strong> (Portugal) and at the Portuguese Geographic Institute, researching<br />

land use and land cover mapping using satellite imagery and developing integrated forest fire risk methods.<br />

Rhiann<strong>on</strong> Gainor is a PhD candidate at McGill University’s School of Informati<strong>on</strong> Studies. A McC<strong>on</strong>nell Foundati<strong>on</strong><br />

Fellow in 2010/2011, her research interests are knowledge management, competitive intelligence, and measurement.<br />

She has a Master’s of Library and Informati<strong>on</strong> Studies, and Master’s of Arts in Humanities Computing from the University<br />

of Alberta.<br />

Audrey Grace has over twelve years of industry experience and is a lecturer with Business Informati<strong>on</strong> Systems at<br />

University College Cork. Her current research interests focus primarily <strong>on</strong> service innovati<strong>on</strong> and the role of informati<strong>on</strong><br />

systems in the delivery of complex services; collaborative systems; learning management systems and<br />

knowledge disseminati<strong>on</strong> within an organisati<strong>on</strong>al c<strong>on</strong>text.<br />

Aida Hadzic is a PhD student at the Department of Applied IT, IT University in Gothenburg. She has a systems science<br />

background and a sec<strong>on</strong>d level educati<strong>on</strong> in the field of IT management. Aida is studying issues related to management<br />

and architectural design of both existing and future IT investments.<br />

Martin Hill has twenty years’ experience building commercial informati<strong>on</strong> systems from Kenyan camel farmers to satellite<br />

operators. Five years developing these for disaster and fr<strong>on</strong>tline military operati<strong>on</strong>s have evolved into academic<br />

research into knowledge distributi<strong>on</strong> across poorly c<strong>on</strong>nected communities<br />

Dr Matthew Hint<strong>on</strong> is Senior Lecturer in Informati<strong>on</strong> <strong>Management</strong> at the Open University Business School. His research<br />

covers the impact of e-commerce <strong>on</strong> operati<strong>on</strong>s and the evaluati<strong>on</strong> of ICT investments. He has published more<br />

than 60 academic articles and two undergraduate teaching texts Introducing Informati<strong>on</strong> <strong>Management</strong>: the Business<br />

Approach and Informati<strong>on</strong> <strong>Management</strong> in C<strong>on</strong>text (2009).<br />

Dr. Vered Holzmann, MBA, is an experienced practicing project manager with a distinguished track record in managing<br />

computer software development teams, implementati<strong>on</strong> of quality assurance programs and management of fast<br />

track c<strong>on</strong>structi<strong>on</strong> projects. She is a faculty member in Hol<strong>on</strong> Institute of Technology - H.I.T. and lectures at Tel-Aviv<br />

University.<br />

Björn Johanss<strong>on</strong> holds a PhD in Informati<strong>on</strong> Systems Development from the Department of <strong>Management</strong> & Engineering<br />

at Linköping University. Currently he works as Associate Senior Lecturer at Department of Informatics, Lund University.<br />

Previously he worked as a Post Doc at the Center for Applied ICT at Copenhagen Business School. He is a<br />

member of the IFIP Working Groups IFIP 8.6 and IFIP 8.9.<br />

Maroun Jneid is a PhD candidate at Université Paris8 with 13 years of professi<strong>on</strong>al experience in software projects<br />

management and engineering process activities and 11 years of experience in software engineering activities lecturing<br />

in the Ant<strong>on</strong>ine University’s Faculty of Engineering as well as its North Campus director for the last 4 years.<br />

Kalimullah Khan a young and energetic student with excellent communicati<strong>on</strong> skills has d<strong>on</strong>e MIT from Pakistan and<br />

an additi<strong>on</strong>al MSc (Thesis resulted awaited) from Blekinge Institute of Technology Sweden. He has an experience in<br />

teaching at university level and wants to pursue his carrier in research (PhD).<br />

Ranjan Kini, Ph.D., is a Professor of Informati<strong>on</strong> Systems. He is an active member of IACIS ACM, DSI, and AIS. His current<br />

research interests include Electr<strong>on</strong>ic and Mobile Commerce, Ethics in Informati<strong>on</strong> Technology, and Health Informati<strong>on</strong><br />

Technology. He is a Senior Editor of Informati<strong>on</strong> Systems <strong>Management</strong> Journal and is also <strong>on</strong> several editorial<br />

boards.<br />

Mike Krey is currently lecturer at Zurich University. Besides his lecturing he is involved in research projects in the field<br />

of Business Integrati<strong>on</strong>. His previous positi<strong>on</strong>s include Business Development Manager for IT Soluti<strong>on</strong>s in the Health<br />

Care Sector. Mike is currently doing his PHD at the Plymouth University in the research field of IT-Governance.<br />

Michael Kyobe is A/Professor of Informati<strong>on</strong> System. He holds a PhD in Computer Informati<strong>on</strong> Systems and an MBA.<br />

Michael worked as a project manager and IT manager for several years and has c<strong>on</strong>sulted extensively with the public,<br />

and SMEs. His research interests include business-IT alignment, governance, computer security, ethics, knowledge<br />

management and SMEs.<br />

x


Alina Madalina L<strong>on</strong>ea is PhD Student at “Politehnica” University of Timisoara (Romania), where she received her BSc<br />

in Systems Engineering and Computers Engineering. She holds a BSc in Computer Science from University of the Highlands<br />

and Islands (UK). Her research interests include Cloud <strong>Management</strong>, Identity Access <strong>Management</strong> and Intrusi<strong>on</strong><br />

Detecti<strong>on</strong> Systems.<br />

Harald Lothaller is employed at the University of Music and Performing Arts Graz (Austria) in two areas. He is head of<br />

the study centre and he is resp<strong>on</strong>sible for statistics, data acquisiti<strong>on</strong>, and the core developer of the presented performance<br />

recording tool. Aside, he is involved in teaching at other HEIs and in research projects.<br />

Carolanne Mah<strong>on</strong>y is a PhD candidate in Business Informati<strong>on</strong> Systems at University College Cork, Ireland. Her research<br />

interests include e-health, informati<strong>on</strong> behaviour, decisi<strong>on</strong> making and supply chain management. She has<br />

previously experience in the electr<strong>on</strong>ics industry.<br />

Nico Martins holds a PhD in Industrial Psychology and is with the Department of Industrial and Organisati<strong>on</strong>al Psychology<br />

at The University of South Africa where he specializes in the field of Organisati<strong>on</strong>al Psychology. He has published<br />

several articles and presented papers at nati<strong>on</strong>al and internati<strong>on</strong>al c<strong>on</strong>ferences in organizati<strong>on</strong>al culture/climate,<br />

organisati<strong>on</strong>al trust, organisati<strong>on</strong>al diagnoses and research .<br />

Laurence Masclet is doing a doctoral research in the field of ethics and regulati<strong>on</strong> of Informati<strong>on</strong> and Communicati<strong>on</strong><br />

Technologies. Her researches take place in the LEGIT laboratory, in Namur University (FUNDP), Belgium. She is a researcher<br />

for the project IDEGOV (Identificati<strong>on</strong> and governance of emerging ethical issues in IS) funded by the CIGREF<br />

foundati<strong>on</strong>.<br />

John McAvoy is a lecturer in the Department of Accountancy Finance and Informati<strong>on</strong> Systems at University College<br />

Cork, Ireland. Prior to lecturing, John had a variety of roles in Informati<strong>on</strong> Systems, ranging from systems administrati<strong>on</strong><br />

to managing software development teams. John has published in a variety of journals and c<strong>on</strong>ferences in the Informati<strong>on</strong><br />

Systems field, c<strong>on</strong>centrating <strong>on</strong> project management, Agile Software Development, and small ISD teams.<br />

Markku Nurminen is Professor Emeritus in Informati<strong>on</strong> Systems at the University of Turku, Finland, EU. He was resp<strong>on</strong>sible<br />

for the introducti<strong>on</strong> of the Master’s Programme in Work Informatics. He has also worked for Universities of<br />

Jyväskylä, Finland, of Bergen and Oslo, Norway. Nurminen is also an active member in the IRIS (Informati<strong>on</strong> Systems<br />

Research in Scandinavia) Associati<strong>on</strong>.<br />

Yv<strong>on</strong>ne O’C<strong>on</strong>nor is a PhD candidate in Business Informati<strong>on</strong> Systems at University College Cork, Ireland. Having a<br />

keen interest in the role Informati<strong>on</strong> Systems play in the healthcare domain, Yv<strong>on</strong>ne’s research explores individual<br />

infusi<strong>on</strong> of mobile health systems in a healthcare envir<strong>on</strong>ment.<br />

Dr. Brian O’Flaherty has lectured in Informati<strong>on</strong> Systems in UCC for over 20 years and holds a PhD in <strong>Management</strong><br />

Science from the University of Strathclyde, Glasgow. He currently focuses <strong>on</strong> teaching technology entrepreneurship<br />

with informati<strong>on</strong> systems. In 2010, Dr O’Flaherty received an award from Enterprise Ireland and Invest NI in recogniti<strong>on</strong><br />

of his c<strong>on</strong>tributi<strong>on</strong> to Technology Entrepreneurship educati<strong>on</strong>.<br />

Paidi O’Raghallaigh is a researcher and part-time lecturer at University College Cork. His academic research primarily<br />

focuses <strong>on</strong> mapping the innovati<strong>on</strong> models of organisati<strong>on</strong>s. He has over 15 years experience as a business and informati<strong>on</strong><br />

systems c<strong>on</strong>sultant and trainer.<br />

R<strong>on</strong>an O’Sullivan is an assistant lecturer in the Business Informati<strong>on</strong> Systems Department at University College Cork.<br />

He is currently working <strong>on</strong> his Ph.D. and his current research interests include service-oriented architecture, web services,<br />

strategic management and business value of IT<br />

Marco Pretorius is the Usability Team Leader at e-Government for Citizens (Western Cape Government). Marco completed<br />

a Masters degree in Computer Science and Informati<strong>on</strong> Technology at the Nels<strong>on</strong> Mandela Metropolitan University<br />

and is a Certified Usability Analyst. He has published in refereed journals and c<strong>on</strong>ference publicati<strong>on</strong>s and his<br />

current PhD research includes usability in e-Government.<br />

Juan Camilo Romero is PhD candidate in Industrial Systems at the University of Toulouse in France. He obtained his<br />

MSc. degree from the same university and has a BSc. degree in Industrial Engineering from the Nati<strong>on</strong>al University of<br />

Colombia. His research interests are focused <strong>on</strong> the problem solving and knowledge management within the frame of<br />

xi


Collaborative Supply Chains. In parallel to his research project, Mr. Romero works as Supply Chain c<strong>on</strong>sultant in the<br />

aer<strong>on</strong>autical industry.<br />

Ted Saarikko is a PhD-student at the Department of Applied IT, IT University in Gothenburg. He has a background in<br />

computer science as well as language studies prior to earning his Masters degree in informatics. Ted is currently studying<br />

issues relating to Enterprise architecture and sustainable development.<br />

Teresa Santos is a post-Doctoural researcher at e-GEO, Centre of Geographical and Regi<strong>on</strong>al Planning Studies, New<br />

University of Lisb<strong>on</strong>, Portugal. She has a master’s in GIS from the Technical University of Lisb<strong>on</strong> (Portugal). Current<br />

research is <strong>on</strong> 3D modelling of geographic data like very high resoluti<strong>on</strong> satellite imagery and LiDAR data, for studying<br />

urban sustainability and land planning.<br />

Gilbert Silvius is professor at HU University of Applied Sciences Utrecht in the Netherlands, where he. is programme<br />

director of the Master of Informatics and Master of Project <strong>Management</strong> programmes. These innovative programmes<br />

link IT and project management to organizati<strong>on</strong>al change and the c<strong>on</strong>cepts of sustainability. Also in research, Gilbert<br />

focuses <strong>on</strong> sustainability in IT, Projects and Project <strong>Management</strong>.<br />

A<strong>on</strong>ghus Sugrue is a PhD candidate in the Accounting, Finance, & Informati<strong>on</strong> Systems department in the University<br />

College of Cork (UCC), Ireland. His research interest focuses up<strong>on</strong> c<strong>on</strong>temporary forms of computer-mediated communicati<strong>on</strong><br />

with particular emphasis <strong>on</strong> the forms of c<strong>on</strong>nectivity affecting individuals both within organisati<strong>on</strong>s and<br />

society in general.<br />

Nestori Syynimaa is a CIO of Anvia Plc, Finland. He holds BBA from Seinäjoki UAS, Finland and MSc <strong>on</strong> Ec<strong>on</strong>omics and<br />

Business Administrati<strong>on</strong> (Computer Science) from University of Vaasa, Finland. He is also a part time PhD student in<br />

Informatics Research Centre, Henley Business School, University of Reading, UK.<br />

Kamila Tislerova, lecturer of Marketing, Faculty of Ec<strong>on</strong>omics, Technical University of Liberec, Czech Republic. Field of<br />

research: Customer Relati<strong>on</strong>ship <strong>Management</strong>; Value of/for customer; e-commerce; all topics also in internati<strong>on</strong>al<br />

dimensi<strong>on</strong>. Visiting lecturer in Universities in Scotland and China. Experience of 15 years in the corporate sector. Interest<br />

in establishing cooperati<strong>on</strong> of business and academic spheres.<br />

Olgerta T<strong>on</strong>a is a PhD Student at Lund University School of Ec<strong>on</strong>omics and <strong>Management</strong>, department of Informatics.<br />

Her current research area is Business Intelligence.<br />

Stephen Treacy graduated from UCC with a Masters in Informati<strong>on</strong> Systems for Business Performance in 2010. Currently<br />

in year two of his PhD course, Stephen is investigating the factors affecting the potential business value of social<br />

media, al<strong>on</strong>g with a proposal of a decisi<strong>on</strong> support framework to identify the key criteria involved therein.<br />

Chris Upfold is a senior lecturer in the Department of Informati<strong>on</strong> Systems at Rhodes University, South Africa. He also<br />

lectures in the Rhodes Business School and for Ernst and Young in South Africa and Mauritius. His areas of interest<br />

and research are Informati<strong>on</strong> Security, Radio Frequency Identificati<strong>on</strong> (RFID), Project <strong>Management</strong>, Virtual Teams and<br />

Corporate Communicati<strong>on</strong>s.<br />

Daryoush Daniel Vaziri received his Bachelor degree from the University of Applied Science in B<strong>on</strong>n-Rhine-Sieg. Afterwards<br />

he worked for Telekom AG. Since 2010 he has been employed at the university of B<strong>on</strong>n-Rhine-Sieg as a research<br />

assistant and simultaneously participating at the Master program “Informati<strong>on</strong> and Innovati<strong>on</strong> management”.<br />

His research fields cover the accessibility of ICT.<br />

Bartosz Wachnik specializes in MIS implementati<strong>on</strong>. He is a member of senior management in Alna Business Soluti<strong>on</strong>s<br />

in Poland, a branch of Lithuanian company, which is <strong>on</strong>e of the largest IT companies in the Baltic area. He has published<br />

more than 20 articles in professi<strong>on</strong>al and academic journals. He has co-operated with University of Technology<br />

in Warsaw where he has PhD.<br />

Haining Wan is a PHD student at School of Informati<strong>on</strong> Systems and <strong>Management</strong>, Nati<strong>on</strong>al University of Defense<br />

Technology, and now a visiting PHD student at Lund University School of Ec<strong>on</strong>omics and <strong>Management</strong>. His current<br />

research interests include: Enterprise Architecture, Business/IT alignment. He got a master degree in Operati<strong>on</strong> Research<br />

(Dec. 2008) and a bachelor degree in Bridge Engineering (Jul. 2006).<br />

xii


Dr. Atieh Zarabzadeh is a post-doctoral researcher in the HISRC, UCC. She holds a PhD in health informatics, TCD. She<br />

is a trained as a Software Engineer, Azzahra University, Tehran, as a result of transferring her IT/Mechanical Engineering<br />

BSc from the Australian Nati<strong>on</strong>al University. Her research interests include applicati<strong>on</strong>s of novel technologies in<br />

healthcare.<br />

Dr Ryszard Zygala is assistant professor at the Wroclaw University of Ec<strong>on</strong>omics, Poland. His research interests include:<br />

informati<strong>on</strong> management, informati<strong>on</strong> systems ec<strong>on</strong>omics, informati<strong>on</strong> systems modeling and architecture. He<br />

has authored a book: Essentials of Business Informati<strong>on</strong> <strong>Management</strong> (in Polish) and c<strong>on</strong>tributed chapters to several<br />

books<br />

xiii


Proposal of Adaptability Indexes to Support <strong>Management</strong><br />

of Engineering and Marketing Systems<br />

Oswaldo Luiz Agostinho<br />

Department of Producti<strong>on</strong> Engineering, São Carlos Engineering School,<br />

University of São Paulo, Brasil<br />

agostinh@sc.usp.br<br />

Abstract: The Resp<strong>on</strong>siveness of Productive Systems has become a prerequisite to meet the demands of high<br />

diversificati<strong>on</strong> of products being manufactured simultaneously, coupled with the reducti<strong>on</strong> of lot sizes and the<br />

reducti<strong>on</strong> of life cycle of those products. Organizati<strong>on</strong>s that have portfolios with large numbers of products and<br />

the ability to absorb shorter product life cycles become competitive against their competitors in the c<strong>on</strong>sumer<br />

market. This article proposes Adaptability Indexes to support the management of the Systems and Processes of<br />

Engineering in organizati<strong>on</strong>s, taking into account the need to reduce design time and the deployment of new<br />

products to meet market demand for these products with increased diversificati<strong>on</strong> and reducti<strong>on</strong> of their useful<br />

life. The indexes of adaptability are defined by the relati<strong>on</strong>ship between time of development of a new product<br />

and the development time of the product immediately subsequent, taking into account a sequence<br />

of development of new products. The time of development of a new product comprises product design,<br />

development of the means of manufacture and c<strong>on</strong>structi<strong>on</strong> of the initial pilot batches. From the definiti<strong>on</strong> of the<br />

adaptability index, engineering organizati<strong>on</strong>s can be classified by the following characteristics: a) regressive<br />

adaptability, when the engineering organizati<strong>on</strong> needs successively greater times in the development of<br />

successive products; b) neutral adaptability, when the engineering organizati<strong>on</strong> requires equivalent development<br />

times for the development of successive products; c) progressive adaptability, when the engineering<br />

organizati<strong>on</strong> successively needs shorter times in the development of successive products. Starting from these<br />

adaptability characteristics of engineering organizati<strong>on</strong>s, the paper discusses the methodologies and<br />

technologies to be applied to products and means of manufacturing development to increase the progressive<br />

indexes of adaptability and reverse engineering organizati<strong>on</strong>s from a regressive adaptability index to a<br />

progressive <strong>on</strong>e. As a c<strong>on</strong>clusi<strong>on</strong>, it will propose that engineering organizati<strong>on</strong>s should have progressive<br />

adaptability indexes in order to increase resp<strong>on</strong>siveness of organizati<strong>on</strong>s that face competitiveness challenges<br />

in envir<strong>on</strong>ments of high diversificati<strong>on</strong> of products and c<strong>on</strong>stant reducti<strong>on</strong> of product life cycle in global markets<br />

in the 21st century. Paper relevance: Resp<strong>on</strong>siveness becomes indispensable for organizati<strong>on</strong>s that compete in<br />

envir<strong>on</strong>ments of products with high diversificati<strong>on</strong> and reducti<strong>on</strong> of life cycles. This paper presents adaptability<br />

indexes to manage Engineering organizati<strong>on</strong>s, to increase their capacity of product and means of manufacturing<br />

design, providing competiveness to attend the external c<strong>on</strong>diti<strong>on</strong>s menti<strong>on</strong>ed above.<br />

Keywords: engineering systems, resp<strong>on</strong>siveness, adaptability, products diversificati<strong>on</strong><br />

1. Introducti<strong>on</strong><br />

Enterprises of the 21 st century are facing competiti<strong>on</strong> <strong>on</strong> various fr<strong>on</strong>ts, such as reducti<strong>on</strong> of product<br />

life time, increase of diversificati<strong>on</strong>, reducti<strong>on</strong> of customer resp<strong>on</strong>se time, internati<strong>on</strong>al competiti<strong>on</strong>.<br />

Organizati<strong>on</strong>s that have portfolios with large numbers of products and the ability to absorb shorter<br />

product life cycles become competitive against their competitors in the c<strong>on</strong>sumer market<br />

(Sch<strong>on</strong>berger, R.J., 2002).<br />

The competitiveness c<strong>on</strong>diti<strong>on</strong>s of an organizati<strong>on</strong> are dependent <strong>on</strong> external influences. One can<br />

classify the external influences as coming from the market, scientific and technological developments,<br />

society and envir<strong>on</strong>ment regulati<strong>on</strong>s. These factors, acting simultaneously, induce a state of external<br />

competitiveness. The external competitiveness is a reference for measuring the internal state of<br />

competitiveness, supported by organizati<strong>on</strong>al, methodological and technological attributes applied<br />

through the activities of the business processes of the organizati<strong>on</strong>. The applicati<strong>on</strong> of technologies<br />

<strong>on</strong> the various business processes, such as shop floor, engineering, support, planning, sales,<br />

purchasing and others, c<strong>on</strong>stitute the technological strategy subordinated to the attributes and aligned<br />

to the respective business strategies. Due to those factors, the Resp<strong>on</strong>siveness of Productive<br />

Systems has become a prerequisite to meet the demands of high diversificati<strong>on</strong> of products being<br />

manufactured simultaneously, coupled with the reducti<strong>on</strong> of lot sizes and the reducti<strong>on</strong> of life cycle of<br />

those products. Resp<strong>on</strong>siveness becomes indispensable for organizati<strong>on</strong>s that compete in<br />

envir<strong>on</strong>ments of products with high diversificati<strong>on</strong> and reducti<strong>on</strong> of life cycles. Looking at these needs,<br />

the Engineering and Marketing organizati<strong>on</strong>s have the relevant resp<strong>on</strong>sibility to meet the<br />

management challenges of paying attenti<strong>on</strong> to the outside market (Porter 2002).<br />

1


2. Objective<br />

Oswaldo Luiz Agostinho<br />

This paper will focus <strong>on</strong> the resp<strong>on</strong>siveness of Engineering and Marketing Systems, named<br />

Adaptability, that provide c<strong>on</strong>diti<strong>on</strong>s for noting external market needs of physical products or services<br />

and the design and manufacturing .<br />

It also proposes adaptability indexes to manage Marketing and Engineering systems in order to<br />

increase the capacity of product design and means of manufacturing, thus providing competiveness<br />

to face the external c<strong>on</strong>diti<strong>on</strong>s menti<strong>on</strong>ed above.<br />

3. Enterprise competitiveness and competitiveness attributes<br />

Enterprise competitiveness can be understood as the capacity to c<strong>on</strong>tinuously review its competiti<strong>on</strong><br />

strategies, obtaining favorable positi<strong>on</strong> in the markets where it is actuated (Agostinho, 2012).<br />

C<strong>on</strong>sequently, it will provide c<strong>on</strong>diti<strong>on</strong>s to generate higher profits than the average of companies of<br />

the market where it is actuated, operating in a sustainable way, with quality, speed and flexibility.<br />

Also, it must satisfy stakeholders and be compliant with envir<strong>on</strong>ment requirements. One can divide<br />

competiveness into two aspects: a) external competiveness, where the enterprise must have the<br />

capacity of provoke in the c<strong>on</strong>sumer the desire of change from a company where he traditi<strong>on</strong>ally buys<br />

a product to the new <strong>on</strong>e. This capacity to drive the so called internal competiveness can be<br />

understood as a set of harm<strong>on</strong>ic and synergic methodological and technological factors that induce<br />

the external market percepti<strong>on</strong> to change products from the previous organizati<strong>on</strong> to the new <strong>on</strong>e.<br />

This state of competiveness is obtained as a c<strong>on</strong>sequence of its organizati<strong>on</strong>al and technological<br />

behaviour, defined as competiveness attributes. These proper characteristics of the management<br />

system are expressed by a c<strong>on</strong>tinuous comprehensive and integrated practice of methodologies, as<br />

part of the organizati<strong>on</strong>al and technological management. The competitiveness attributes and subattributes<br />

are classified according their range of applicati<strong>on</strong>:<br />

Market driven attributes: Proper characteristics of the management system of the organizati<strong>on</strong><br />

expressed by the c<strong>on</strong>tinuous, comprehensive and integrated use of methodologies driven to provide<br />

c<strong>on</strong>diti<strong>on</strong>s of attendance of needs of the c<strong>on</strong>sumer markets. The market driven attributes are<br />

deployed in the following sub attributes:<br />

a) Innovati<strong>on</strong>: The process of technological innovati<strong>on</strong> comprises a complex set of activities that<br />

transform ideas and scientific knowledge into physical reality and real world applicati<strong>on</strong>s. It also<br />

integrates existing technology and inventi<strong>on</strong>s to create a new product, service or process. Innovative<br />

companies look for ideas of any kind, and provide an organizati<strong>on</strong>al culture that supports their<br />

development in viable business programmes. The innovati<strong>on</strong> sub-attribute can be split: a) focus <strong>on</strong><br />

c<strong>on</strong>sumer needs; b) enterprise wide quality; c) enterprise wide planning; d) utilizati<strong>on</strong> of core<br />

knowledge; e) c<strong>on</strong>tinuous upgrading of products and processes.<br />

b) Agility: Enterprise agility can be reached when the technological and administrative infrastructure is<br />

flexible, and can be rapidly created, c<strong>on</strong>figured and rearranged, meeting external business needs.<br />

Am<strong>on</strong>g others, this attribute facilitates a superior time to market for new business initiatives. The<br />

agility sub- attribute can be split: a) independence from technology; b) re-use of soluti<strong>on</strong>s;<br />

c) adequate business infrastructure; d) resp<strong>on</strong>se capacity of the whole system, not just isolated<br />

processes; e) existence and applicati<strong>on</strong> of strategies; f) extended enterprise business architecture<br />

that includes supply chain and marketing processes.<br />

c) Resp<strong>on</strong>siveness: The characteristic of the organizati<strong>on</strong> to resp<strong>on</strong>d to the external needs in an<br />

adequate time, also called “time to market“. These characteristics include pro-active search of<br />

feedback from customers and suppliers, the ability to be a rapid cycle organizati<strong>on</strong>, flexible in thinking<br />

and doing things in short periods of time. The resp<strong>on</strong>siveness sub-attributes can be split: a) Business<br />

process optimizati<strong>on</strong>, with eliminati<strong>on</strong> of the activities that do not add value to the product or to the<br />

business; b) adopti<strong>on</strong> of proven technology; c) business strategies integrati<strong>on</strong>; d) Resp<strong>on</strong>se time from<br />

the Engineering and Marketing processes; e) optimizati<strong>on</strong> of the supply chain network.<br />

Organizati<strong>on</strong> driven attributes: Proper characteristics of the organizati<strong>on</strong> management system<br />

expressed by c<strong>on</strong>tinuous, comprehensive and integrated methodologies driven to provide c<strong>on</strong>diti<strong>on</strong>s<br />

for organizati<strong>on</strong>al structures to take note of the external needs that determine the external<br />

competitiveness. The organizati<strong>on</strong> driven attributes determine the way that the administrative and<br />

2


Oswaldo Luiz Agostinho<br />

management business processes are applied throughout the organizati<strong>on</strong> itself. The organizati<strong>on</strong><br />

driven attributes can be split: a) coexistence between the hierarchical and business process<br />

organizati<strong>on</strong> models; b) informati<strong>on</strong> infrastructure; c) knowledge management; d) organizati<strong>on</strong><br />

c<strong>on</strong>trols.<br />

Human capital driven attributes: Proper characteristics of the management system of the organizati<strong>on</strong><br />

expressed by c<strong>on</strong>tinuous, comprehensive and integrated methodologies for the development of<br />

programmes of selecti<strong>on</strong>, educati<strong>on</strong>, and training of the human capital that supports the organizati<strong>on</strong><br />

to reach an organizati<strong>on</strong>al level that leads to the competitiveness status. The human capital driven<br />

attributes can be split: a) teamwork; b) project clusters; c) human networking d) hierarchy versus<br />

participati<strong>on</strong>.<br />

4. Organizati<strong>on</strong> resp<strong>on</strong>siveness<br />

The objective of this paper is to show that the development of the resp<strong>on</strong>siveness of the organizati<strong>on</strong><br />

can be split into two characteristics, the so-called adaptability of the Engineering and Marketing<br />

Systems, and the flexibility of the Shop Floor systems.<br />

Organizati<strong>on</strong> Resp<strong>on</strong>siveness can be defined as the Manufacturing system capacity to implement<br />

changes in adequate times that meet the needs of the c<strong>on</strong>sumer market, mainly in the development of a<br />

new product or simultaneously manufacturing different parts or quantities in the shop floor envir<strong>on</strong>ment<br />

(Agostinho, 2011). This paper will deal with the Resp<strong>on</strong>siveness of the Marketing and Engineering<br />

systems to meet the external market needs of products diversificati<strong>on</strong>, reducti<strong>on</strong> of products life cycle,<br />

competiti<strong>on</strong> internati<strong>on</strong>alizati<strong>on</strong>.<br />

5. Adaptability of marketing and engineering systems<br />

The management approach to the Marketing and Engineering Systems must take into c<strong>on</strong>siderati<strong>on</strong><br />

the need to achieve competitiveness attributes to meet the external demands of reducti<strong>on</strong> of the<br />

useful life of products, increase product diversificati<strong>on</strong>, the internati<strong>on</strong>alizati<strong>on</strong> of markets, and<br />

reducti<strong>on</strong> of time to reach markets with new products, mainly in the years 2000.( W. H. El Maraghy,<br />

2007). Looking at the approach above, this paper will focus <strong>on</strong> the resp<strong>on</strong>siveness resp<strong>on</strong>sible for<br />

detecting external market needs of physical products or services, their product design, manufacturing<br />

specificati<strong>on</strong>s development and pre-producti<strong>on</strong>, defined as Adaptability of Engineering and Marketing<br />

Systems. This will be defined as Adaptability of the Marketing and Engineering Systems as the<br />

capacity of the Marketing and Engineering Systems introduce simultaneously and successively new<br />

products in adequate times, to meet the needs of c<strong>on</strong>sumer market (Agostinho, 2011).<br />

To better develop the adaptability c<strong>on</strong>cept, it will be necessary to define the activities related to the<br />

introducti<strong>on</strong> of new products by the organizati<strong>on</strong>. Table 1 details the main activities and sub activities<br />

related to the implementati<strong>on</strong> of a new product, for the functi<strong>on</strong>s of Marketing, Engineering and<br />

Producti<strong>on</strong> (Chang, 2005).<br />

Table 1: Main activities of product development (Chang, 2005)<br />

Functi<strong>on</strong> Activity Activity detailment<br />

Marketing Product acceptance research C<strong>on</strong>sumer market needs<br />

Definiti<strong>on</strong> of product c<strong>on</strong>cept<br />

Engineering Product design<br />

Manufacturing specificati<strong>on</strong>s<br />

development<br />

Implementati<strong>on</strong> time frame<br />

C<strong>on</strong>ceptual design<br />

Dimensi<strong>on</strong>ing<br />

Parts detail<br />

Functi<strong>on</strong>al tests<br />

Reliability tests<br />

Manufacturing routing<br />

Manufacturing Process detail<br />

Tooling and machine tools<br />

Manufacturing times<br />

Producti<strong>on</strong>/Engineering Pre Producti<strong>on</strong> Tooling try out<br />

Manufacturing processes try out<br />

Pilot run<br />

For each activity, there is an associated respective executi<strong>on</strong> time and corresp<strong>on</strong>dent angle (figure 1).<br />

3


T f<br />

Total time<br />

product<br />

development.<br />

θ<br />

T p<br />

T pa<br />

T pr<br />

T mf<br />

θ<br />

pa<br />

Product<br />

acceptance<br />

research<br />

Oswaldo Luiz Agostinho<br />

Product<br />

design<br />

θ<br />

p<br />

mf<br />

Manufacturing<br />

specificati<strong>on</strong>s<br />

development<br />

Product devellopment<br />

Figure 1: Activities of a new product development (Agostinho, 2011)<br />

- Time of product acceptance research<br />

Tpa<br />

Tp<br />

Tmf<br />

Tpr<br />

- Time of product design<br />

- Time of manufacturing specificati<strong>on</strong>s development<br />

- Time of pre-producti<strong>on</strong><br />

Tf Total time product development.<br />

θpa<br />

θp<br />

θmf<br />

θpr<br />

θf<br />

- product acceptance research.<br />

- product development<br />

- development of manufacturing specificati<strong>on</strong>s<br />

- pre–producti<strong>on</strong><br />

- total product development<br />

θf<br />

θ<br />

θ<br />

pr<br />

Pre-Producti<strong>on</strong><br />

Af<br />

Activities<br />

It can be observed that the angles θpa, θp , θmp , θpr , and c<strong>on</strong>sequently, θf , are functi<strong>on</strong>s of the<br />

relati<strong>on</strong>ship activities x time per activity. Each activity and respective time can have a different behaviour,<br />

depending <strong>on</strong> the degree of organizati<strong>on</strong> of Marketing, Product Engineering and Manufacturing<br />

Engineering Systems.<br />

The product implementati<strong>on</strong> total time Tf is expressed as:<br />

Tf = Tpa + Tp +Tmf +Tpr ,<br />

4


5.1 Adaptability indexes<br />

Oswaldo Luiz Agostinho<br />

In order to provide management c<strong>on</strong>diti<strong>on</strong>s for the Marketing and Engineering organizati<strong>on</strong>s and drive<br />

them to meet the organizati<strong>on</strong>s need to be aware of the external market demands; this work will<br />

define the numerical indexes of adaptability. Supposing that an organizati<strong>on</strong> needs to launch new<br />

products, starting from product 1 up to product n in a defined period of time, after the development of<br />

a product i , assumed as a reference, products i+1 and i+2 will be developed successively, with the<br />

same activities Af. This can be observed in figure 2:<br />

Total<br />

development<br />

time<br />

Tfi + 1<br />

Tfi<br />

Tfi + 2<br />

0<br />

Θi<br />

Θi+2<br />

Θi+1<br />

Activities<br />

Figure 2: Adaptability of engineering and marketing systems (Agostinho, 2011)<br />

In figure 2 can be seen:<br />

Tfi – Time to develop product i;<br />

Tf i+1 – Time to develop product i+1 ;<br />

Tfi+2 – Time to develop product i+2<br />

Where Tf i+1 < Ti , and Tf i+2 > Ti<br />

Af<br />

Product i +1<br />

Product i<br />

Product i +2<br />

The Adaptability of the Engineering and Marketing systems to introduce the product i+ 1 after the<br />

introducti<strong>on</strong> of the product i , or to introduce the product i +2 after the introducti<strong>on</strong> of product i is<br />

expressed by:<br />

Generalizing, supposing that an Engineering and Marketing organizati<strong>on</strong> needs to introduce a<br />

sequence of products, from product 1 to product n, it will define adaptability of the Marketing and<br />

Engineering systems to introduce a product i+1, after the introducti<strong>on</strong> of product i as :<br />

5<br />

> 1<br />

< 1


Oswaldo Luiz Agostinho<br />

The relati<strong>on</strong>ship above defines numerically the adaptability indexes of the Marketing and Engineering<br />

Systems, varying as follows:<br />

∞ > a i,,i+1 ≥ 0<br />

5.2 Analysis of marketing and engineering systems using adaptability indexes<br />

Analyzing the relati<strong>on</strong>ship above, <strong>on</strong>e can classify:<br />

Neutral Adaptability: Neutral adaptability occurs when ai, i + 1 =1, i.e. the Engineering and<br />

Marketing systems introduce a new product with equivalent times to the introducti<strong>on</strong> of the<br />

previous product or Ti + 1 = Ti. This situati<strong>on</strong> occurs predominantly in organizati<strong>on</strong>s that have<br />

products with very stable design and l<strong>on</strong>g useful life, low diversificati<strong>on</strong>, low external demands<br />

from the c<strong>on</strong>sumer market. Normally these kinds of organizati<strong>on</strong>s are classified within the<br />

m<strong>on</strong>opolistic markets, or with very stable products in their portfolios.<br />

Regressive Adaptability: Regressive adaptability occurs when ai, i + 1 < 1, i.e. the Engineering<br />

and Marketing Systems introduce a new product with times Ti + 1 higher then the times of the<br />

previous product ( Ti ). The limit value ai, i + 1 = 0 will occur when the Marketing and Engineering<br />

Systems need time tending to infinite to introduce a new product. This limit situati<strong>on</strong> shows<br />

complete inadequacy of the Marketing and Engineering Systems to face the challenges of<br />

introducti<strong>on</strong> of new products. This situati<strong>on</strong> occurs in organizati<strong>on</strong>s where the Engineering and<br />

Marketing systems are poorly organized, with very poor product design criteria and respective<br />

data bases, processes with activities that do not add value, lack of utilizati<strong>on</strong> of parts grouping in<br />

families, lack of use of methodologies of activities superpositi<strong>on</strong>, like C<strong>on</strong>current Engineering,<br />

random use of automati<strong>on</strong>, such as CAD, CAPP, CAM, local networks, etc . This level of<br />

Marketing and Engineering systems will not allow the organizati<strong>on</strong> to compete in aggressive<br />

markets, but just in niche markets, where competitiveness is not so necessary, and the product<br />

brand is a guarantee of sales, even in poor c<strong>on</strong>diti<strong>on</strong>s.<br />

Progressive Adaptability: Progressive adaptability occurs when ai, i + 1 > 1, i.e. the Marketing and<br />

Engineering Systems introduce a new product with times Ti + 1 smaller than the times of the<br />

previous product Ti. The limit value will occur when ai, i + 1 = ∞, i.e. the Marketing and<br />

Engineering Systems need times close to zero to introduce a new product after the introducti<strong>on</strong><br />

of the previous <strong>on</strong>e. In this limit situati<strong>on</strong>, the introducti<strong>on</strong> of new products is made almost<br />

instantaneously, dem<strong>on</strong>strating that the Marketing and Engineering Systems are adequate and<br />

able to resp<strong>on</strong>d to any challenge coming from the c<strong>on</strong>sumer market. This situati<strong>on</strong> occurs in<br />

organizati<strong>on</strong>s where the Engineering and Marketing systems are very well organized, both in the<br />

product design criteria, as the type of organizati<strong>on</strong> that is implemented. The next paragraph<br />

details the main criteria to provide c<strong>on</strong>stant increase of the adaptability indexes. Progressive<br />

adaptability provides c<strong>on</strong>diti<strong>on</strong>s to keep the organizati<strong>on</strong> at an adequate competitiveness level to<br />

face the demand challenges of the aggressive external market c<strong>on</strong>diti<strong>on</strong>s of the 21 st century.<br />

6. Criteria to increase adaptability<br />

The current c<strong>on</strong>diti<strong>on</strong>s of the c<strong>on</strong>sumer market indicate that the c<strong>on</strong>stant changes of products with<br />

increasingly shorter life times are very demanding to maintain competitiveness. To meet this demand,<br />

it becomes necessary for the Marketing and Engineering systems to maintain progressive adaptability<br />

indexes. So, in order to keep ai, i + 1 > 1 , it will be necessary to accomplish management and<br />

technological c<strong>on</strong>diti<strong>on</strong>s in the product design and manufacturing specificati<strong>on</strong>s development as<br />

follows.<br />

6.1 Reducti<strong>on</strong> of complexity<br />

The reducti<strong>on</strong> of complexity of designed products and manufacturing specificati<strong>on</strong>s c<strong>on</strong>tributes to<br />

increased adaptability because simpler projects can be optimized and reproduced, simplifying the<br />

products data base of standard parts. The reducti<strong>on</strong> of complexity of products can be obtained<br />

through reducti<strong>on</strong> of parts, part forms simplificati<strong>on</strong> to facilitate the manufacturing routing and<br />

manufacturing process, careful evaluati<strong>on</strong> of specificati<strong>on</strong>s of dimensi<strong>on</strong>al, geometric and surface<br />

roughness tolerances, materials and surface and heat treatments. Looking at the manufacturing side,<br />

the reducti<strong>on</strong> of complexity is achieved through the standardizati<strong>on</strong> of manufacturing routes, reducti<strong>on</strong><br />

of operati<strong>on</strong>s derived from the evaluati<strong>on</strong> of parts specificati<strong>on</strong>s, standardizati<strong>on</strong> of tooling design,<br />

6


Oswaldo Luiz Agostinho<br />

reducti<strong>on</strong> or eliminati<strong>on</strong> of unnecessary operati<strong>on</strong>s due to choosing inadequate machine tools that do<br />

not meet the specificati<strong>on</strong>s of dimensi<strong>on</strong>al, geometric or metallurgical deviati<strong>on</strong>s (Nam P. Suh, 2008).<br />

6.2 Project parts standardizati<strong>on</strong><br />

The standardizati<strong>on</strong> of comp<strong>on</strong>ents in several projects, with re-utilizati<strong>on</strong> of existing parts greatly<br />

reduces the development time of a new project (Tarek, 2003). The applicati<strong>on</strong> of the Group<br />

Technology methodology, supported by an efficient data base of similar parts, will make possible:<br />

Grouping the parts in families, to increase project rati<strong>on</strong>alizati<strong>on</strong>;<br />

Retrieving existing informati<strong>on</strong> related to existing projects, to be applied to new projects;<br />

Standardizati<strong>on</strong> of specificati<strong>on</strong>s, characteristics and materials;<br />

Upgrading products with the eliminati<strong>on</strong> of duplicate drawings;<br />

Development of standard routings and manufacturing processes, derived from part families;<br />

Standardizati<strong>on</strong> of tooling derived from the standard manufacturing processes.<br />

6.3 Optimizati<strong>on</strong> of the marketing and engineering systems<br />

Looking at the Marketing and Engineering systems, the reducti<strong>on</strong> in number of necessary activities to<br />

develop a new product can be obtained by two different ways:<br />

a) Reducti<strong>on</strong> of sub-activities that comprise the engineering and marketing business processes,<br />

through the critical analysis of their real need. The applied methodology is Business Processes<br />

Reengineering, with the eliminati<strong>on</strong> of activities that do not add value to the product or business. As<br />

an example, is the activity of checking drawings really necessary? Are there ways to increase project<br />

robustness by avoiding unnecessary checking <strong>on</strong> the drawings? To achieve these objectives,<br />

methodologies must be applied of standardizati<strong>on</strong> and simplificati<strong>on</strong> of product design and<br />

manufacturing processes sheets, with the applicati<strong>on</strong> of Group Technology, Design for Manufacturing,<br />

am<strong>on</strong>g others.<br />

b) Superpositi<strong>on</strong> or eliminati<strong>on</strong> of activities: the eliminati<strong>on</strong> of activities can be effective after the<br />

applicati<strong>on</strong> of item 6.1. The other alternative will be the superpositi<strong>on</strong> of activities, changing from<br />

serial activities to parallel or simultaneous activities, applying the C<strong>on</strong>current Engineering<br />

methodology.<br />

It can be observed that, besides the time for product development and manufacturing specificati<strong>on</strong>s<br />

being greater when C<strong>on</strong>current Engineering is applied, the total product development time will be<br />

less, due to the superpositi<strong>on</strong> of the activities (Chang, 2005).<br />

6.4 Reducti<strong>on</strong> of time per activity<br />

The reducti<strong>on</strong> of the time per activity will be achieved by the utilizati<strong>on</strong> of Informati<strong>on</strong> Technology<br />

resources, such as CAD, CAE, CAPP, CAM, Stress modelling softwares, etc. The computati<strong>on</strong>al<br />

resources will be applied after the previous criteria of adaptability increase have already been applied,<br />

as described above: Reducti<strong>on</strong> of complexity, Projects parts standardizati<strong>on</strong> Optimizati<strong>on</strong> of the<br />

Marketing and Engineering organizati<strong>on</strong>s through Business Processes Reengineering and C<strong>on</strong>current<br />

Engineering.<br />

7. C<strong>on</strong>clusi<strong>on</strong>s<br />

Enterprises in the 21 st century are facing competiti<strong>on</strong> <strong>on</strong> various fr<strong>on</strong>ts, such as reducti<strong>on</strong> of product<br />

life time, increase of diversificati<strong>on</strong>, reducti<strong>on</strong> of customer resp<strong>on</strong>se time, competiti<strong>on</strong><br />

internati<strong>on</strong>alizati<strong>on</strong>. Organizati<strong>on</strong>s that have portfolios with large numbers of products and the ability<br />

to absorb shorter product life cycles become competitive against their competitors in the c<strong>on</strong>sumer<br />

market. Marketing and Engineering systems are of great importance in maintaining c<strong>on</strong>diti<strong>on</strong>s to<br />

compete in these c<strong>on</strong>stantly changing envir<strong>on</strong>ments, enabling organizati<strong>on</strong>s to provide the necessary<br />

time to market for new products (Sluga, 2008). In order to provide management c<strong>on</strong>diti<strong>on</strong>s to manage<br />

the Marketing and Engineering organizati<strong>on</strong>s and drive them to meet the organizati<strong>on</strong>s’ need to be aware<br />

of external market needs, this work proposes numerical indexes of adaptability.<br />

7


Oswaldo Luiz Agostinho<br />

In order to maintain the c<strong>on</strong>diti<strong>on</strong>s of competitiveness, mainly related to paying attenti<strong>on</strong> to external<br />

market needs, such as reducti<strong>on</strong> of time to market, increase of diversificati<strong>on</strong>, reducti<strong>on</strong> of useful life of<br />

products, adaptability must be maintained progressively, i.e. a i,i+1 > 1. To keep it in this c<strong>on</strong>diti<strong>on</strong>, the<br />

several methodologies described above must be applied in the business processes of both Marketing<br />

and Engineering systems. The m<strong>on</strong>itoring of the adaptability indexes, with the purpose of maintaining<br />

progressive adaptability is a pre-c<strong>on</strong>diti<strong>on</strong> to be competitive, providing the c<strong>on</strong>sumer market with always<br />

new products with the necessary diversificati<strong>on</strong>.<br />

References<br />

Agostinho, O.L. (2011) Manufacturing Systems (in Portuguese), Faculdade de Engenharia Mecanica,<br />

Universidade Estadual de Campinas, São Paulo, Brasil.<br />

Agostinho, O.L., Batocchio, A., Silva (2012) Proposal of Methodology to Balance, Correlate and Align Technology<br />

and Business Strategies to Competitiveness Organizati<strong>on</strong> Attributes, PMA 2012 <str<strong>on</strong>g>C<strong>on</strong>ference</str<strong>on</strong>g>, Cambridge,<br />

UK.<br />

Agostinho, O.L. (2008) Technology and Business Strategies: Methodology for correlati<strong>on</strong> using knowledge<br />

management, Internati<strong>on</strong>al C<strong>on</strong>gress <strong>on</strong> <strong>Management</strong> of Technology (IAMOT), Nice, France.<br />

Chang, C.M. (2005) Engineering <strong>Management</strong> – Challenges in the New Millennium, Pears<strong>on</strong>/Prentice Hall, N.J..<br />

USA .<br />

El Maraghy, W.H., Urbanic, R.J. (2007) Modelling of Manufacturing Systems Complexity, Annals of CIRP.<br />

<str<strong>on</strong>g>European</str<strong>on</strong>g> Institute for Technology and Innovati<strong>on</strong> <strong>Management</strong> (2004) Bringing Technology and Innovati<strong>on</strong> into<br />

the Boardroom, Palgrave Macmillan, L<strong>on</strong>d<strong>on</strong>, UK.<br />

Fine, C.H. (2001) Clock Speed – Winning Industry C<strong>on</strong>trol in the Age of Temporary Advantage, Perseus Books,<br />

Bost<strong>on</strong>, USA.<br />

Madapusi, A. and Miles, G. (2011) Routines in enterprise applicati<strong>on</strong> systems; <strong>Management</strong> Research Review.<br />

2011, Vol. 34, 1, pp. 75-97.<br />

Nam P. Suh, (2008) Complexity in Engineering, Annals of CIRP.<br />

Porter, M.E., Van der Linde, C. (2002) Toward a new c<strong>on</strong>cepti<strong>on</strong> of the envir<strong>on</strong>ment-competitiveness<br />

relati<strong>on</strong>ship? In: Stavins, R. (Ed.). Ec<strong>on</strong>omics of the envir<strong>on</strong>ment: selected readings. New York: W. W.<br />

Nort<strong>on</strong> & Company.<br />

Sch<strong>on</strong>berger, R.J. (2002) World Class Manufacturing – The Next Decade, The Free Press, N.Y., USA.<br />

Sluga, P. Butala1, J. Peklenik1 ((2008) A C<strong>on</strong>ceptual Framework for Collaborative Design and Operati<strong>on</strong>s of<br />

Manufacturing Work Systems, Annals of CIRP.<br />

Tarek, K.M. (2003) <strong>Management</strong> of Technology, McGraw Hill, N.Y., USA .<br />

8


Knowledge Gaps in Post-Merger Integrati<strong>on</strong> of Software<br />

Maintenance Processes: A Case Study<br />

Maria Alaranta and Eero Martela<br />

Department of Industrial Engineering and <strong>Management</strong>, Aalto University,<br />

Espoo, Finland<br />

maria.alaranta@aalto.fi<br />

emartela@gmail.com<br />

Abstract: The integrati<strong>on</strong> activities after a merger or an acquisiti<strong>on</strong> result in and are troubled by loss of<br />

knowledge that is critical for running the business. However, we know little of such knowledge gaps. We adapt<br />

the IS Body of Knowledge by Iivari et al., 2004 to form a framework for knowledge gaps in post-merger<br />

integrati<strong>on</strong> of software processes. The key knowledge types are: C<strong>on</strong>text-of-use knowledge, softwareengineering<br />

knowledge, organizati<strong>on</strong>al knowledge, transformati<strong>on</strong> knowledge and operative knowledge. Our<br />

empirical data comes from the integrati<strong>on</strong> of the software maintenance processes after a megamerger in the ICT<br />

field. We explore what types of knowledge loss trouble the integrati<strong>on</strong> and when such losses are most prominent.<br />

In the case studied, all five types of knowledge gaps are present. Key findings include the variety and persistence<br />

of knowledge problems and their evoluti<strong>on</strong>ary nature. There are gaps in several types of knowledge at all<br />

integrati<strong>on</strong> phases and all types of knowledge during ramp-up. Gaps in transformati<strong>on</strong> knowledge persist<br />

throughout the integrati<strong>on</strong> process. Knowledge gaps evolve over time both in terms of which gaps are prominent<br />

at each phase of the change, and in terms of the specific c<strong>on</strong>tent of the gaps. The framework allows to<br />

systematically integrate insights provided by prior literature, and it can help researchers of knowledge issues<br />

during turbulent times to focus their studies. Our analysis also reveals the emergent nature of knowledge-related<br />

problems in post-merger process integrati<strong>on</strong>. Researchers would benefit from including this evoluti<strong>on</strong>ary<br />

perspective in their studies, and thus being able to build more descriptive theories. Practiti<strong>on</strong>ers can use our<br />

results to identify, understand and prepare for specific knowledge-loss problems, thus facilitating overcoming<br />

these problems and managing risks. They may also benefit from engaging in <strong>on</strong>-going creati<strong>on</strong> and transfer of<br />

critical knowledge during all phases of integrati<strong>on</strong>.<br />

Keywords: knowledge gap, M&A, merger, post-merger integrati<strong>on</strong>, software maintenance<br />

1. Introducti<strong>on</strong><br />

Mergers and acquisiti<strong>on</strong>s (M&As) are dramatic events in a company’s life cycle that have fundamental<br />

influences <strong>on</strong> its business processes. They are am<strong>on</strong>g key strategy tools. Yet, 50 – 80 percent of the<br />

deals fail to meet their targets (Alaranta & Kautz, forthcoming). Am<strong>on</strong>g main causes for high failure<br />

rates is integrati<strong>on</strong> problems; including poor integrati<strong>on</strong> of knowledge resources (Yoo et al. 2007). As<br />

a result, much of the knowledge that is critical for running the business processes is lost.<br />

This loss of knowledge disrupts the sequence of related business processes that together result as a<br />

product or service. These processes include both the operati<strong>on</strong>al activities (e.g., research, analysis,<br />

design, coding and support testing) and managerial activities (coordinating and managing) (Cf. Grant,<br />

1996, Cha et al. 2008). The capability to perform such activities depends largely <strong>on</strong> a firm’s prior<br />

experience <strong>on</strong> such tasks, i.e., learning-by-doing knowledge. (Cha et al., 2008)<br />

Knowledge gaps refer to situati<strong>on</strong>s in which an organizati<strong>on</strong> or <strong>on</strong>e part of it needs some knowledge<br />

but is for some reas<strong>on</strong> lacking it (Cha et al., 2008). Any of a company’s knowledge processes –<br />

knowledge creati<strong>on</strong>, knowledge storage/retrieval, knowledge transfer and knowledge applicati<strong>on</strong><br />

(Alavi et al., 2001) – may be disrupted resulting in a knowledge gap. In a merger, an organizati<strong>on</strong>’s<br />

knowledge processes are disrupted, i.e. some knowledge may be missing from some parts of the<br />

organizati<strong>on</strong> and some knowledge isn’t transferring due to the sudden loss of employees’<br />

c<strong>on</strong>necti<strong>on</strong>s. In additi<strong>on</strong>, managers’ tendency to stick to old processes and ways of operating may<br />

further amplify the disrupti<strong>on</strong> of knowledge chains as the knowledge c<strong>on</strong>figurati<strong>on</strong>s aren’t adapted to<br />

the new situati<strong>on</strong>. Due to these reas<strong>on</strong>s, a merger causes gaps in knowledge related to where to<br />

acquire needed knowledge. (Yoo et al. 2007) Furthermore, a comm<strong>on</strong> reas<strong>on</strong> for the existence of<br />

knowledge gaps is that some knowledge exists within the organizati<strong>on</strong> but the part of organizati<strong>on</strong><br />

that would need the knowledge doesn’t know about the availability of that knowledge (Alavi et al.,<br />

2001; Szulanski, 2000) or where to acquire that knowledge (Alavi et al., 2001; Yoo et al., 2007).<br />

The empirical c<strong>on</strong>text for this study is corrective software maintenance process. Software<br />

maintenance states for ‘the modificati<strong>on</strong> of a software product after delivery, to correct faults, to<br />

9


Maria Alaranta and Eero Martela<br />

improve performance or other attributes, or to adapt the product to a modified envir<strong>on</strong>ment.’ (IEEE,<br />

1998) Software maintenance is an interesting c<strong>on</strong>text for this study for its ec<strong>on</strong>omic importance - the<br />

cost of software maintenance has been estimated to have raised from approximately 35% of IS costs<br />

in 1960s to up to 90% in 1990’s (Polo et al., 2003) - and knowledge-intensive nature (Anquetil et al.,<br />

2006). Software maintenance plagued by inadequate informati<strong>on</strong> and uncertainties (Zmud, 1980). In<br />

corrective software maintenance, some key sources of knowledge gaps include poor documentati<strong>on</strong>,<br />

schedule problems, interdependencies with other comp<strong>on</strong>ents in the product in questi<strong>on</strong> (Lientz et al.,<br />

1978) as well as problems related to communicati<strong>on</strong>, interacti<strong>on</strong> and interfaces, and user involvement<br />

(Kajko-Mattss<strong>on</strong>, 2004).<br />

In this research, we examine and illustrate the knowledge gaps in an empirical case of integrating<br />

software maintenance processes after a merger. The empirical research questi<strong>on</strong> is: “What<br />

knowledge gaps occur in post-merger integrati<strong>on</strong> of software maintenance processes and how do<br />

knowledge gaps change over time?”<br />

2. Background<br />

Prior research <strong>on</strong> the issues of knowledge processes in M&As provides useful initial insights<br />

regarding the nature of knowledge gaps. Yoo et al. (2007) explored the fracturing of knowledge<br />

c<strong>on</strong>figurati<strong>on</strong>s after a merger. They discuss the process of creating a mutual stock of knowledge<br />

am<strong>on</strong>g groups or individuals, focusing <strong>on</strong> the operative knowledge needed to carry out daily activities.<br />

Emps<strong>on</strong> (2001) discusses the meaning of organizati<strong>on</strong>s’ knowledge bases for the success of postmerger<br />

knowledge transfer. Her focus is <strong>on</strong> the role that individuals play in the transfer of technical,<br />

organizati<strong>on</strong>al and client c<strong>on</strong>text knowledge. Bresman et al. (1999) promoted the significance of<br />

pers<strong>on</strong>al c<strong>on</strong>tacts in the transfer of technological know-how and patents after a merger.<br />

In order to analyze the knowledge gaps in post-merger integrati<strong>on</strong>, we need framework for organizing<br />

relevant knowledge. As the focus of our empirical study is in software maintenance processes, we<br />

adopt the systematic body of knowledge (BoK) for IS development (c<strong>on</strong>sisting of system<br />

development, system operati<strong>on</strong> and system maintenance) by Iivari et al (2004). The IS BoK is a<br />

suitable base for our analysis as such process knowledge allows us to organize the practically<br />

relevant IS knowledge in an acti<strong>on</strong>-oriented way. We build our analysis <strong>on</strong> its five knowledge areas:<br />

technical knowledge, applicati<strong>on</strong> domain knowledge, organizati<strong>on</strong>al knowledge, IS applicati<strong>on</strong><br />

knowledge, and IS development process knowledge (Iivari et al., 2004).<br />

Technical knowledge encompasses the knowledge related to the types of hardware and software as<br />

well as their applicati<strong>on</strong> (Iivari et al., 2004). In the c<strong>on</strong>text of post-merger integrati<strong>on</strong> of software<br />

maintenance processes, this translates into the technical knowledge needed by an engineer to carry<br />

out daily software maintenance tasks which we term Engineering knowledge. Applicati<strong>on</strong> domain<br />

knowledge covers the knowledge about the applicati<strong>on</strong> domain – such as accounting c<strong>on</strong>cepts and<br />

principles - for which an informati<strong>on</strong> system is built (Iivari et al., 2004). The applicati<strong>on</strong> of this in the<br />

c<strong>on</strong>text of post-merger integrati<strong>on</strong> is operative knowledge, i.e., the n<strong>on</strong>-technical knowledge needed<br />

to carry out the daily activities. This includes the knowledge that is needed for daily tasks such as<br />

comm<strong>on</strong> working habits and employee networks. According to Iivari et al. (2004), Systems<br />

development process knowledge refers to the tools, techniques, methods, approaches and principles<br />

used in developing IS. In post-merger integrati<strong>on</strong>, what is developed is the new organizati<strong>on</strong>al<br />

c<strong>on</strong>figurati<strong>on</strong>. Thus, we need transformati<strong>on</strong> knowledge i.e., knowledge about how the organizati<strong>on</strong>s<br />

merge; such as planning and ramping up the integrati<strong>on</strong> or catalyzing the cooperati<strong>on</strong> of employees<br />

from different backgrounds. Organizati<strong>on</strong>al knowledge refers to the knowledge <strong>on</strong> the organizati<strong>on</strong><br />

including its work processes, structure, culture, role distributi<strong>on</strong>, cause-effect relati<strong>on</strong>ships, flow of<br />

informati<strong>on</strong> etc. (Cf. Iivari et al., 2004). Finally, IS applicati<strong>on</strong> knowledge includes <strong>on</strong> how an IS may<br />

support the activities in its intra- and inter-organizati<strong>on</strong>al c<strong>on</strong>text. (Iivari et al., 2004). We broaden this<br />

c<strong>on</strong>cept to include all relevant knowledge related to the c<strong>on</strong>text in which the software maintenance<br />

process takes place, such as knowledge about customers’ expectati<strong>on</strong>s or customer’s business and<br />

call it C<strong>on</strong>text-of-use Knowledge.<br />

The following table summarizes the related c<strong>on</strong>tributi<strong>on</strong>s of existing literature in relati<strong>on</strong> to the five<br />

knowledge types relevant to post-merger integrati<strong>on</strong> of software maintenance processes.<br />

10


Maria Alaranta and Eero Martela<br />

Table 1: Framework for understanding knowledge types and examples in prior literature<br />

Knowledge type Examples in prior literature <strong>on</strong> postmerger<br />

integrati<strong>on</strong><br />

C<strong>on</strong>text-of-use - Client knowledge: general<br />

knowledge understanding of a particular industry;<br />

detailed knowledge of a specific client<br />

firm; and pers<strong>on</strong>al knowledge of key<br />

individuals within the client firm.<br />

(Emps<strong>on</strong>, 2001)<br />

Engineering - Individual technical knowledge<br />

knowledge<br />

(Emps<strong>on</strong>, 2001)<br />

- Engineering know-how and knowwhat<br />

(Bresman et al., 1999)<br />

- Informati<strong>on</strong> systems and data bases<br />

c<strong>on</strong>sisting of technical data (Yoo et al.<br />

2007)<br />

Operative - knowledge-sharing resources and<br />

knowledge<br />

practices (Yoo et al. 2007)<br />

(Youngjin et al., 2007)<br />

Organizati<strong>on</strong>al<br />

knowledge<br />

Transformati<strong>on</strong><br />

knowledge<br />

- Sectoral knowledge (Emps<strong>on</strong>, 2001)<br />

- Firm-specific organizati<strong>on</strong>al<br />

knowledge (Emps<strong>on</strong>, 2001)<br />

- c<strong>on</strong>necti<strong>on</strong>s between people (Yoo et<br />

al. 2007)<br />

- Prior post-acquisiti<strong>on</strong> IS integrati<strong>on</strong><br />

experience; post-acquisiti<strong>on</strong> IS<br />

integrati<strong>on</strong> skills (Alaranta & Kautz,<br />

forthcoming)<br />

Examples of knowledge gaps in the empirical<br />

data<br />

“What the customer wants to know definitely<br />

after an outage or in an emergency [is] …<br />

what the actual root cause was. How can I<br />

avoid … in the future? We are very bad in<br />

communicating root causes for our<br />

customers.”<br />

“… we haven’t succeeded in transferring the<br />

work from the fr<strong>on</strong>t-end into the middle level<br />

according to the original plans.”<br />

“So it was not answering ph<strong>on</strong>e calls, not<br />

answering email, not cooperating, not really<br />

understanding, I guess, why they should be<br />

doing this.”<br />

“… we had two different people from two<br />

different organizati<strong>on</strong>s, and they weren’t that<br />

close, you know, just with how they were<br />

working.”<br />

“Sure, I guess there was resistance, because<br />

they were uncomfortable doing it, didn’t<br />

understand the value, the benefits, and hadn’t<br />

been told they had to do it.”<br />

The body of practically relevant knowledge <strong>on</strong> post-merger integrati<strong>on</strong> is highly dispersed. Prior<br />

literature addresses what knowledge is transferred, transformed or recreated. The how the<br />

transformati<strong>on</strong> is carried out is <strong>on</strong>ly discussed in the c<strong>on</strong>text of IS integrati<strong>on</strong>. In additi<strong>on</strong>, n<strong>on</strong>e of the<br />

prior c<strong>on</strong>tributi<strong>on</strong>s touches all five areas of knowledge. We complement these studies by addressing<br />

also the transformati<strong>on</strong> knowledge <strong>on</strong> the organizati<strong>on</strong>al level and by integrating the five knowledge<br />

areas.<br />

3. Methodological choices<br />

Post-merger integrati<strong>on</strong> as a phenomen<strong>on</strong> has several features that limit the choice of effective data<br />

collecti<strong>on</strong> methods, such as unavailability of informati<strong>on</strong> <strong>on</strong> future mergers prior to closing the deal as<br />

well as high uncertainty and sensitive nature of the mergers. These unique pressures make for<br />

example surveys and ethnographies are oftentimes inc<strong>on</strong>ceivable. (Yoo et al. 2007)<br />

Case research was chosen for this study because it allows in-depth understanding of real-life<br />

situati<strong>on</strong>s. The qualitative method allowed us to access and understand fine-grained issues related to<br />

both merging parties and all relevant actors. A study of a single case is an appropriate strategy for<br />

revelatory studies (Yin, 1984). Megamergers such as the <strong>on</strong>e studied in this paper are rare but<br />

dramatic events in companies’ life cycles. They offer unique opportunities for observing major<br />

knowledge-process changes and disrupti<strong>on</strong>s.<br />

The chosen merger case, Alpha-Beta (pseud<strong>on</strong>ym), is interesting for this study because; first, the<br />

merger deal had been signed two years prior to the beginning of this research project; i.e., we could<br />

retrospectively access the life cycle of the post-merger integrati<strong>on</strong> and the events of the integrati<strong>on</strong><br />

project were still in fresh memory of the employees. Sec<strong>on</strong>d, the chosen case is <strong>on</strong>e of merger of<br />

equals in which full integrati<strong>on</strong> of the software maintenance processes was desired. We sought such<br />

case to ensure that we could observe a range of problems related to disrupti<strong>on</strong> of existing knowledge<br />

processes and c<strong>on</strong>structing new <strong>on</strong>es. Third, to further amplify the visibility of knowledge issues in<br />

post-merger integrati<strong>on</strong>, we chose a knowledge-intensive process for our study.<br />

We chose semi-structured interviews because of their ability to provide versatile enough data about<br />

the phenomen<strong>on</strong> and leave enough flexibility for each stakeholder to describe their experiences (Cf.<br />

Yin, 1984). The initial list of interviewees was composed in collaborati<strong>on</strong> with the case organizati<strong>on</strong> to<br />

cover both merging organizati<strong>on</strong>s (14 interviewees from ex-Alpha and 9 interviewees from ex-Beta)<br />

11


Maria Alaranta and Eero Martela<br />

as well as all relevant organizati<strong>on</strong>al units and levels. A few new interviewees were added based <strong>on</strong><br />

suggesti<strong>on</strong>s received during the interviews, and follow-up interviews were carried out with two key<br />

actors <strong>on</strong>e year after c<strong>on</strong>ducting this study. Each interview lasted 1-2 hrs and the interviews were<br />

tape-recorded and later transcribed.<br />

The interview themes included:<br />

Resp<strong>on</strong>dent background<br />

The post-merger integrati<strong>on</strong> of corrective software maintenance process phase by phase: What?<br />

Who? How? Why?<br />

Less<strong>on</strong>s learned?<br />

To c<strong>on</strong>firm, compare and c<strong>on</strong>trast the data collected via the interviews, we also used documents<br />

received from the case company. These included process models, project plans, and organizati<strong>on</strong><br />

charts. (Cf. Yin, 1984)<br />

Data analysis: In this study, data collecti<strong>on</strong> and analysis were intertwined. 2-3 interviewers were<br />

present at each interview, and the interviewees discussed emergent themes after each interview.<br />

Additi<strong>on</strong>al questi<strong>on</strong>s were included in the interview guide to either c<strong>on</strong>firm or deny emergent themes.<br />

This paper presents the results related to <strong>on</strong>e of the themes that emerged during this study, namely<br />

knowledge gaps. These policies served to improve the validity of this research (Cf. Yin, 1984).<br />

As we explored the theme of knowledge gaps empirically, we also simultaneously compared our<br />

findings to relevant former research (Strauss & Corbin, 1998), and developed the coding scheme. Our<br />

analysis proceeded in three phases. First, a narrative including key acti<strong>on</strong>s and events was<br />

composed (see Secti<strong>on</strong> 0). Sec<strong>on</strong>d, we coded the data for the five categories of knowledge gaps. We<br />

operated <strong>on</strong> the <strong>on</strong>tological assumpti<strong>on</strong>s that knowledge gaps are evident from problems, rework,<br />

delays, etc. as well as indirectly when organizati<strong>on</strong>al actors employed various techniques for<br />

overcoming knowledge gaps. These included e.g., virtual organizati<strong>on</strong>s, recruiting, communicati<strong>on</strong>s<br />

and training. We also attributed a phase <strong>on</strong> the post-merger time line for each knowledge gap. Third,<br />

we re-interpreted the case data from the perspective of the knowledge gaps. This allowed us to gain a<br />

rich understanding of the nature and timing of knowledge gaps in post-merger integrati<strong>on</strong> of software<br />

maintenance processes.<br />

4. The empirical case<br />

4.1 Case setting<br />

This case study c<strong>on</strong>cerns the integrati<strong>on</strong> of two very distinct software maintenance processes as a<br />

result of a mega-merger of two multinati<strong>on</strong>al ICT companies. The newly formed Alpha-Beta<br />

(pseud<strong>on</strong>ym) operates in business-to-business markets, and its products c<strong>on</strong>sist of hardware and<br />

software comp<strong>on</strong>ents. The motive for this merger was to achieve ec<strong>on</strong>omies of scale and synergy<br />

savings in highly competitive markets.<br />

Both companies had headquarters in Europe and employed over 40 000 people. Beta’s organizati<strong>on</strong>al<br />

culture was very hierarchical meanwhile Alpha drove a more informal culture that relied <strong>on</strong> informal<br />

relati<strong>on</strong>ships.<br />

The focus of this study is the post-merger integrati<strong>on</strong> of the corrective software maintenance<br />

processes. Such process starts when a customer informs the company of a problem and ends when a<br />

soluti<strong>on</strong> is provided to the customer. Before the merger, Alpha’s run this process as a geographically<br />

scattered but globally managed network and it had a centralized global IS support for the process.<br />

Beta, in turn, had organized its regi<strong>on</strong>ally managed process into a hierarchy around its centralized<br />

back-end units, and it ran a scattered IS landscape with various IS in different geographical locati<strong>on</strong>s.<br />

Alphabeta’s process integrati<strong>on</strong> proceeded through three distinct phases:<br />

Planning: Integrati<strong>on</strong>-related acti<strong>on</strong>s executed before the day <strong>on</strong>e of the merger<br />

Ramp-up: Building comm<strong>on</strong> operati<strong>on</strong>s after the day <strong>on</strong>e of the merger<br />

Refining: Implementing corrective acti<strong>on</strong>s after reviewing outcomes of the ramp-up<br />

12


4.2 Findings<br />

Maria Alaranta and Eero Martela<br />

Planning: The initial knowledge gaps were related to what would be the principles and culture of the<br />

new organizati<strong>on</strong> and how to proceed with merging the companies. In additi<strong>on</strong>, the employees were<br />

largely ignorant of the future of their positi<strong>on</strong>s.<br />

The management’s lack of knowledge about how to integrate was, first, due to their unfamiliarity with<br />

the counterpart organizati<strong>on</strong>s. One software maintenance manager told us: ”the challenge was<br />

always to [know] … how many [maintenance] cases for which technology coming from which<br />

geographical area. … we had data from ex-Alpha, data from ex-Beta, there were not, let's say,<br />

c<strong>on</strong>sistent instructi<strong>on</strong>s or there was a lot of anticipati<strong>on</strong>, and just guessing how will the number of<br />

cases develop.” Without knowing the structures and figures of the other organizati<strong>on</strong>, the<br />

management couldn’t plan an optimal model for integrati<strong>on</strong>. To overcome this, a project structure was<br />

established for integrating the two former corrective software-maintenance processes into <strong>on</strong>e. The<br />

initial task of the project teams was to collect and combine knowledge from the two organizati<strong>on</strong>s. The<br />

management also lacked experience and knowledge about how these kinds of integrati<strong>on</strong>s should be<br />

executed and of ideal models for these kinds of processes. They studied literature and hired<br />

c<strong>on</strong>sultants to overcome this gap.<br />

Due to the different backgrounds, there was a huge gap in old and new cultures and principles of<br />

operati<strong>on</strong>. This caused c<strong>on</strong>fusi<strong>on</strong> and slowed down decisi<strong>on</strong>-making. An ex-Alpha manager<br />

complained: “When I try to reach my supervisor, his assistant answers me. And then I’m c<strong>on</strong>fused<br />

because I’d like to call my supervisor directly. Luckily, for years, nobody [in ex-Alpha] has had to<br />

reach anybody through an assistant. … In ex-Alpha we had this joke about management by SMS. But<br />

then I got these colleagues who hadn’t sent any SMS in their life.” The management attempted to<br />

attenuate these tensi<strong>on</strong>s by communicating and openly discussing differences in the cultures between<br />

the merging organizati<strong>on</strong>s as well as by communicating the values of the new firm.<br />

Due to the lack of knowledge, many employees felt their positi<strong>on</strong>s and future in the company<br />

threatened causing employee turnover as well as general uncertainty, as a customer engagement<br />

manager stated “The company [should have expressed] a clear commitment … to certain areas of the<br />

business. Not general messages, like we did … There should be some areas where engineers could<br />

feel completely safe. We have an extreme amount of engineers that have left the company. … They<br />

were afraid that they would be fired. … Often these engineers are the most competent resources that<br />

we have.” As a soluti<strong>on</strong>, the management aimed to make fast decisi<strong>on</strong>s at the managerial level and<br />

limiting the amount of changes made. The key outcome of the planning stage was a blueprint of the<br />

new process.<br />

Ramp-up: One year after the signing of the deal, the merger was closed and Alpha-Beta set to<br />

execute the integrati<strong>on</strong>. The executi<strong>on</strong> team travelled to the regi<strong>on</strong>s in order to communicate the<br />

changes and jointly define the details of the changes. The largest sub-projects were relocating<br />

pers<strong>on</strong>nel to the new, centralized service centers and the implementati<strong>on</strong> of a new, integrated<br />

informati<strong>on</strong> system. The implementati<strong>on</strong> of centralized service centers caused a temporary gap in<br />

engineering knowledge in the new locati<strong>on</strong>s. In the beginning, the new sited didn’t have enough<br />

human resources both in terms of head count and engineering skills. As a result, the maintenance<br />

process couldn’t run as planned. The executi<strong>on</strong> team needed to “… identify what gaps there were in<br />

… training requirements or even skills development, but mostly <strong>on</strong> what [the engineers] were doing<br />

today compared to what they will need to do tomorrow, and… how do you fill that competence gap to<br />

enable this.”, as <strong>on</strong>e team member explained. The gaps were slowly overcome through internal<br />

recruiting and training as well as using virtual teams during the transiti<strong>on</strong> period.<br />

The merger-related structural changes caused gaps in operative knowledge, organizati<strong>on</strong>al<br />

knowledge, and c<strong>on</strong>text-of-use knowledge. Operative knowledge didn’t flow between the process<br />

levels because engineers <strong>on</strong> various levels resisted the change. A member of the executi<strong>on</strong> team<br />

complained: “I sensed very clearly from day <strong>on</strong>e direct resistance from ex-Beta to move over to ex-<br />

Alpha processes. ... So just very direct ‘No!’ We’re not going to do it.” The resistance manifested in<br />

reluctance to forward cases to subsequent process levels and distrust between them. As <strong>on</strong>e<br />

software maintenance manager complained: “there’s a huge gap between the back-end and middlelevel,<br />

like the Berlin wall”.<br />

13


Maria Alaranta and Eero Martela<br />

In additi<strong>on</strong>, the implementati<strong>on</strong> of the new process-support IS was delayed. This, in turn, delayed the<br />

process integrati<strong>on</strong> as some regi<strong>on</strong>s had to c<strong>on</strong>tinue using their previous IS, and the management<br />

lacked an overall visi<strong>on</strong> to the process during the delay. Temporarily, this caused a severe knowledge<br />

gap for the managers: they were not aware of the poor process performance.<br />

The engineers resisted the new IS due to its poor usability. They found creative ways to bypass it,<br />

which caused further gaps in the operative knowledge. In additi<strong>on</strong>, the case flow in the new process<br />

took place via the new IS <strong>on</strong>ly. However, this lack of pers<strong>on</strong>al interacti<strong>on</strong>s between the engineers<br />

decreased the efficiency in sharing knowledge. The management took steps to improve the IS and<br />

their communicati<strong>on</strong> to alleviate these problems.<br />

One severe problem during this phase was that the customer percepti<strong>on</strong> suffered. This was due to an<br />

excessive focusing <strong>on</strong> internal changes and sub-optimizing within process levels. The management<br />

attempted to tackle this problem by encouraging more communicati<strong>on</strong> with the customer and between<br />

process levels. Customer feedback was collected to get a better understanding of the problems<br />

Refining: Over <strong>on</strong>e and a half years after the merger, the managers c<strong>on</strong>sidered the main integrati<strong>on</strong><br />

effort finished, and they initiated various projects for further developing the process. The manager of<br />

<strong>on</strong>e maintenance unit described this situati<strong>on</strong>: “Once you survive day <strong>on</strong>e and the car is driving more<br />

or less into right directi<strong>on</strong>, immediately take the next step and try to review what you have now. …<br />

See what we have, how it works, analyze the weak points, hear the customer expectati<strong>on</strong>s, … and<br />

take corrective acti<strong>on</strong>s as fast as possible. … we have a lot of good things and lots of gaps, so after<br />

the compromises [in choosing the best practices from Ex-Alpha and Ex-Beta] we have to … correct<br />

the directi<strong>on</strong>. ” Again, the managers had to overcome the gap of what and how to develop. They<br />

c<strong>on</strong>ducted a thorough analysis of the situati<strong>on</strong> and launched projects to improve operative knowledge<br />

and c<strong>on</strong>text-of-use knowledge. These projects included: implementati<strong>on</strong> of formal operative<br />

methodologies, boosting IS capabilities, improving case routing, a service excellence c<strong>on</strong>cept, and<br />

planning to rec<strong>on</strong>figure the process flow.<br />

The following<br />

Table 2 summarizes the key knowledge gaps Apha-Beta needed to overcome to integrate the<br />

software maintenance processes.<br />

Table 2: Knowledge gaps to be overcome in Alpha-Beta’s integrati<strong>on</strong> of software-maintenance<br />

Planning Transformati<strong>on</strong> knowledge<br />

- gaps in what and how to integrate; know the future c<strong>on</strong>figurati<strong>on</strong> of the organizati<strong>on</strong><br />

Organizati<strong>on</strong>al knowledge<br />

- gaps in comm<strong>on</strong> ways of working or comm<strong>on</strong> culture<br />

Ramp-up Operative knowledge<br />

- gaps in knowledge <strong>on</strong> the new maintenance process<br />

Engineering knowledge<br />

- gaps in engineering skills in the new units<br />

Organizati<strong>on</strong>al knowledge<br />

- before the IS integrati<strong>on</strong>, gaps in visibility to process performance<br />

C<strong>on</strong>text of use<br />

- gaps in knowledge <strong>on</strong> customers’ problems and needs due to transformati<strong>on</strong> work<br />

Transformati<strong>on</strong> knowledge<br />

-gaps in how to implement the integrati<strong>on</strong>-related changes in different units<br />

Refining Operative knowledge<br />

- gaps in how to further improve the maintenance process<br />

C<strong>on</strong>text of use knowledge<br />

- gaps in knowledge <strong>on</strong> customers’ problems and needs<br />

Transformati<strong>on</strong> knowledge<br />

- gaps in knowledge <strong>on</strong> how to further develop the process<br />

5. Discussi<strong>on</strong><br />

Various and persistent knowledge problems: There were gaps in every knowledge area,<br />

especially during ramp-up. This shows how a merger fundamentally disrupts the entire knowledge<br />

c<strong>on</strong>figurati<strong>on</strong> of an organizati<strong>on</strong>. Yet, c<strong>on</strong>tributi<strong>on</strong>s in existing literature covers <strong>on</strong>ly <strong>on</strong>e (Bresman et<br />

14


Maria Alaranta and Eero Martela<br />

al., 1999) or few (Emps<strong>on</strong>, 2001; Yoo et al., 2007) knowledge areas at time, and the nature of<br />

transformati<strong>on</strong> knowledge is <strong>on</strong>ly accounted for IS issues. In additi<strong>on</strong>, there are gaps in<br />

transformati<strong>on</strong> knowledge at every stage of the post-merger integrati<strong>on</strong> process. This finding is in line<br />

with the noti<strong>on</strong> that post-merger integrati<strong>on</strong>s are complex (Alaranta & Kautz, forthcoming), and it<br />

reveals a key challenge: transformati<strong>on</strong> knowledge must be created for each unique merger situati<strong>on</strong><br />

by combining knowledge <strong>on</strong> both organizati<strong>on</strong>s and desired goals.<br />

Whilst the prominence of c<strong>on</strong>textual organizati<strong>on</strong>al knowledge in post-merger integrati<strong>on</strong> is not new to<br />

the literature (Emps<strong>on</strong>, 2001; Yoo et al., 2007), what is perhaps new is that problems related to it<br />

seem to attenuate towards the end of the integrati<strong>on</strong> life cycle. This may be because collaborati<strong>on</strong><br />

increases mutual understanding of each other’s’ c<strong>on</strong>text (Cf. Bresman et al., 1999). In additi<strong>on</strong>, some<br />

gaps in organizati<strong>on</strong>al knowledge were diminished via the implementati<strong>on</strong> of a comm<strong>on</strong> IS (Cf.<br />

Alaranta & Kautz, forthcoming)<br />

Alpha-Beta’s the biggest problems related to performance and customer satisfacti<strong>on</strong> seem to stem<br />

from gaps in transformati<strong>on</strong> knowledge, organizati<strong>on</strong>al knowledge and c<strong>on</strong>text-of-use knowledge.<br />

Problems in engineering knowledge (the c<strong>on</strong>tent of the maintenance process) are prominent <strong>on</strong>ly<br />

during ramp up but all other knowledge types cause major problems in at least two phases. This is in<br />

line with prior literature in the sense that the transfer of explicit engineering knowledge is easier than<br />

the transfer and creati<strong>on</strong> of the other, more tacit types of knowledge (Cf. N<strong>on</strong>aka, 1994).<br />

Evoluti<strong>on</strong>ary knowledge problems: There is an evoluti<strong>on</strong> in both the prominence of the different<br />

knowledge gaps and their c<strong>on</strong>tents. Whilst prior research acknowledges the emergent nature of postmerger<br />

integrati<strong>on</strong> (Alaranta & Kautz, forthcoming), our study is the first <strong>on</strong>e to take the knowledge<br />

perspective, providing insights into both causes and dynamics of this evoluti<strong>on</strong>.<br />

There are also interdependencies between the knowledge gaps. The heavy loads work related to<br />

creating transformati<strong>on</strong> and organizati<strong>on</strong>al knowledge caused loss of c<strong>on</strong>text-of-use knowledge<br />

during ramp-up and refining, even though both ex-organizati<strong>on</strong>s had efficient processes for creating<br />

and transferring it. This insight is in line with prior post-merger integrati<strong>on</strong> research that predicts value<br />

destructi<strong>on</strong> in such change processes (Haspeslagh & Jemis<strong>on</strong>, 1991). What is perhaps new is<br />

showing that knowledge-related problems may directly c<strong>on</strong>tribute to this value destructi<strong>on</strong>.<br />

6. C<strong>on</strong>clusi<strong>on</strong>s<br />

This paper addresses knowledge gaps in post-merger integrati<strong>on</strong> of software maintenance processes.<br />

Prior research predicts value destructi<strong>on</strong> in post-merger integrati<strong>on</strong>s (Haspeslagh & Jemis<strong>on</strong>, 1991),<br />

and our research shows that knowledge-related problems directly c<strong>on</strong>tribute to it. We adapt a<br />

framework for understanding the nature of such knowledge loss. We integrate existing c<strong>on</strong>tributi<strong>on</strong>s<br />

<strong>on</strong> what knowledge needs to be transferred, transformed or recreated, and add transformati<strong>on</strong><br />

knowledge, i.e., knowledge <strong>on</strong> how the integrati<strong>on</strong> is carried out. Our empirical study also addresses<br />

when during the post-merger transformati<strong>on</strong> is each knowledge type most pr<strong>on</strong>e to gaps.<br />

This initial analysis can serve as a resource for researchers and managers. Our findings show that<br />

scrutinizing different types of knowledge gaps at all phases of the integrati<strong>on</strong> process is a key to<br />

fruitful analyses and successful management of post-merger situati<strong>on</strong>s. The framework can be used<br />

to focus future studies <strong>on</strong> IT during turbulent times.<br />

Researchers could also benefit from including the evoluti<strong>on</strong>ary perspective in their studies, and thus<br />

being able to build more descriptive theories. Future research could also focus <strong>on</strong> practices for<br />

overcoming knowledge gaps during turbulent times; including transferring knowledge across units<br />

(Szulanski, 2000), and creating new knowledge (N<strong>on</strong>aka, 1994).<br />

The framework and the empirical analysis can support practiti<strong>on</strong>ers’ risk analyses and provide cues<br />

<strong>on</strong> what knowledge needs to be transferred or created to overcome in their merger cases.<br />

Practiti<strong>on</strong>ers may also take home the noti<strong>on</strong> that merging organizati<strong>on</strong>s benefit from engaging in <strong>on</strong>going<br />

creati<strong>on</strong> and transfer of relevant knowledge. This is particularly true for transformati<strong>on</strong><br />

knowledge.<br />

15


References<br />

Maria Alaranta and Eero Martela<br />

Alaranta, Maria – Kautz, KarlHeinz (forthcoming). A Framework for Understanding Post-Acquisiti<strong>on</strong> IS<br />

Integrati<strong>on</strong>. Journal of Informati<strong>on</strong> Technology Theory and Applicati<strong>on</strong><br />

Alavi, M., & Leidner, D. E. 2001. Review: Knowledge <strong>Management</strong> and Knowledge <strong>Management</strong> Systems:<br />

C<strong>on</strong>ceptual Foundati<strong>on</strong>s and Research Issues. MIS Quarterly, 25(1): 107-136.<br />

Anquetil, N., Oliveira, K., & Dias, M. 2006. Software maintenance <strong>on</strong>tology. Ontologies for Software Engineering<br />

and Software Technology: 153-173.<br />

Bresman, H., Birkinshaw, J., & Nobel, R. 1999. Knowledge Transfer in Internati<strong>on</strong>al Acquisiti<strong>on</strong>s. Journal of<br />

Internati<strong>on</strong>al Business Studies, 30(3): 439-462.<br />

Cha, H. S., Pingry, D. E., & Thatcher, M. E. 2008. Managing The Knowledge Supply Chain: An Organizati<strong>on</strong>al<br />

Learning Model Of Informati<strong>on</strong> Technology Offshore Outsourcing. MIS Quarterly, 32(2): 281-306.<br />

Emps<strong>on</strong>, L. 2001. Fear of Exploitati<strong>on</strong> and Fear of C<strong>on</strong>taminati<strong>on</strong>: Impediments to Knowledge Transfer in<br />

Mergers between Professi<strong>on</strong>al Service Firms. Human Relati<strong>on</strong>s, 54(7): 839-862.<br />

Grant, R. M. 1996. Toward a Knowledge-Based Theory of the Firm. Strategic <strong>Management</strong> Journal, 17(Winter):<br />

14.<br />

Haspeslagh, P., & Jemis<strong>on</strong>, D. 1991. Managing acquisiti<strong>on</strong>s: Creating value through corporate renewal: The Free<br />

Press New York.<br />

IEEE. 1998. IEEE. Standard for Software Maintenance, 1998; 47 pp.<br />

Iivari, J., Hirschheim, R., & Klein, H. 2004. Towards a distinctive body of knowledge for Informati<strong>on</strong> Systems<br />

experts: coding ISD process knowledge in two IS journals. Informati<strong>on</strong> Systems Journal, 14(4): 313-342.<br />

Kajko-Mattss<strong>on</strong>, M. 2004. Problems within fr<strong>on</strong>t-end support. Journal of Software Maintenance and Evoluti<strong>on</strong>:<br />

Research and Practice, 16(4-5): 309-329.<br />

Lientz, B. P., Swans<strong>on</strong>, E. B., & Tompkins, G. E. 1978. Characteristics of applicati<strong>on</strong> software maintenance.<br />

Commun. ACM, 21(6): 466-471.<br />

N<strong>on</strong>aka, I. 1994. A Dynamic Theory of Organizati<strong>on</strong>al Knowledge Creati<strong>on</strong>. Organizati<strong>on</strong> Science, 5(1): 14-37.<br />

Polo, M., Piattini, M., & Ruiz, F. 2003. Advances in software maintenance management: Technologies and<br />

soluti<strong>on</strong>s: IGI Global.<br />

Strauss, A., & Corbin, J. 1998. Basics of Qualitative Research: Techniques and Procedures for Developing<br />

Grounded Theory (2 ed.). Thousand Oaks, CA: SAGE Publicati<strong>on</strong>s.<br />

Szulanski, G. 2000. The Process of Knowledge Transfer: A Diachr<strong>on</strong>ic Analysis of Stickiness. Organizati<strong>on</strong>al<br />

Behavior and Human Decisi<strong>on</strong> Processes, 82(1): 9-27.<br />

Yin, R. K. 1984. Case Study Research - Design and Methods. Newbury Park, CA, USA: SAGE.<br />

Yoo, Y., Lyytinen, K., & Heo, D. 2007. Closing the gap: towards a process model of post-merger knowledge<br />

sharing. Informati<strong>on</strong> Systems Journal, 17(4): 321-347.<br />

Zmud, R. W. 1980. <strong>Management</strong> Of Large Software Development Efforts. MIS Quarterly, 4(2): 45-55.<br />

16


A Proposed Framework for Guiding the Effective<br />

Implementati<strong>on</strong> of an Informal Communicati<strong>on</strong> System for<br />

Virtual Teams<br />

Garth Alistoun and Christopher Upfold<br />

Rhodes University, Grahamstown, South Africa<br />

Galistoun@gmail.com<br />

c.upfold@ru.ac.za<br />

Abstract: This research provides insight into the nature of informal communicati<strong>on</strong>, and how it relates to virtual<br />

teams. It is shown that informal communicati<strong>on</strong> plays a critical role in the achievement of the producti<strong>on</strong>, group<br />

maintenance and member support goals of a team. Given the dispersed nature of virtual teams, it is shown that<br />

the lack of access to face-to-face communicati<strong>on</strong> results in challenges to effective teamwork. These challenges<br />

are (1) Trust building, (2) Informati<strong>on</strong> exchange, (3) Process gains and losses, (4) Feelings of isolati<strong>on</strong>, (5)<br />

Participati<strong>on</strong>, (6) Coordinati<strong>on</strong> and (7) Cohesi<strong>on</strong>. In order to overcome these challenges, five functi<strong>on</strong>al needs of<br />

a virtual informal communicati<strong>on</strong> system are identified. These needs are (1) Co-presence, (2) Low behavioural<br />

cost, (3) Visual channel, (4) Document sharing and (5) Multiple complimentary systems. The proposed framework<br />

of an effective virtual communicati<strong>on</strong> system is intended to provide guidelines against which intended or existing<br />

virtual communicati<strong>on</strong> systems can be assessed. Given that the framework is theoretical, further research will be<br />

required to determine its veracity.<br />

Keywords: virtual teams, globalisati<strong>on</strong>, communicati<strong>on</strong>, distributed teams<br />

1. Introducti<strong>on</strong><br />

Barriers of distance are shrinking due to ever improving communicati<strong>on</strong> technologies. Humankind is<br />

at any moment c<strong>on</strong>nected through teleph<strong>on</strong>es, email, Voice over IP and a host of related<br />

technologies. Businesses and other organisati<strong>on</strong>s are increasingly taking advantage of these<br />

technologies which are being used to underpin both local and global collaborati<strong>on</strong>. Project teams are<br />

now often made up of members who are geographically distributed and are forced to use<br />

technological communicati<strong>on</strong> mediums to co-ordinate and complete their work. Numerous studies<br />

have been c<strong>on</strong>ducted to develop a model of virtual team communicati<strong>on</strong> processes and the means of<br />

improving these interacti<strong>on</strong>s through communicati<strong>on</strong> technologies (Maruping and Agarwal, 2004;<br />

Kirkman, Rosen, Gibs<strong>on</strong>, Tesluk and McPhers<strong>on</strong>, 2002; Geister, K<strong>on</strong>radt and Hertel, 2006; Liu and<br />

Burn, 2007). This paper focuses <strong>on</strong> the relatively new area of virtual collaborati<strong>on</strong> research, the role<br />

informal communicati<strong>on</strong> plays in virtual teams. The majority of studies and communicati<strong>on</strong><br />

technologies discount or ignore the value of informal communicati<strong>on</strong> in virtual teams in favour of a<br />

more structured, task-based approach to team development and maintenance, at least in the early<br />

stages of a virtual team’s lifecycle (Kirkman et al., 2002; 70-71; Whittaker, Frohlich and Daly-J<strong>on</strong>es,<br />

1994: 131; Liu and Burn, 2007: 48-49, Pins<strong>on</strong>neault and Caya, 2005: 10). Given this apparent neglect<br />

of such a key form of communicati<strong>on</strong> (Kraut et al., 1990: 15; Isaacs et al., 1997: 2), there is a need to<br />

develop technologies that enable virtual group members to maintain their relati<strong>on</strong>ships, co-ordinate<br />

their work and succeed in the most natural way possible.<br />

2. Informal communicati<strong>on</strong><br />

Informal communicati<strong>on</strong> is a sp<strong>on</strong>taneous communicative event between random, out of role<br />

participants who do not prearrange the topic of c<strong>on</strong>versati<strong>on</strong> (Kraut et al. 1990a: 5). The informal<br />

communicati<strong>on</strong> process is also highly interactive, c<strong>on</strong>tent rich and informal in terms of speech register<br />

and language usage (Kraut et al. 1990a: 5) and is performed synchr<strong>on</strong>ously in face-to-face settings<br />

(Whittaker et al., 1994: 131). Kraut et al. (1990a: 15) provide a tax<strong>on</strong>omy by which c<strong>on</strong>versati<strong>on</strong>s can<br />

be classified. These categories are (1) scheduled, (2) intended, (3) opportunistic and (4)<br />

sp<strong>on</strong>taneous. Scheduled c<strong>on</strong>versati<strong>on</strong>s have prearranged c<strong>on</strong>tent and c<strong>on</strong>text (Kraut et al., 1990a:<br />

15), intended c<strong>on</strong>versati<strong>on</strong>s are those in which the initiating participant specifically goes in search of<br />

the other party (Kraut et al., 1990a: 15), opportunistic c<strong>on</strong>versati<strong>on</strong>s are categorized as those in which<br />

the initiating party has planned to talk with the other participant at some point and has taken<br />

advantage of a chance encounter to have the c<strong>on</strong>versati<strong>on</strong> (Kraut et al., 1990a: 15), and sp<strong>on</strong>taneous<br />

interacti<strong>on</strong>s are in no way pre-planned by the participants. Isaacs et al. (1997: 5) regard intended,<br />

opportunistic and sp<strong>on</strong>taneous interacti<strong>on</strong>s as informal in the way they are initiated.<br />

17


Garth Alistoun and Christopher Upfold<br />

Informal communicati<strong>on</strong> plays an important role in supporting effective teamwork through workexecuti<strong>on</strong>,<br />

group co-ordinati<strong>on</strong>, socializati<strong>on</strong> of team members and team building processes<br />

(Whittaker et al., 1994: 131). Isaacs et al. (1997: 9) state that the restricti<strong>on</strong> of these communicati<strong>on</strong><br />

processes, such as in distributed collaborati<strong>on</strong>, has a negative impact <strong>on</strong> task-performance despite a<br />

greater emphasis placed <strong>on</strong> formal meetings.<br />

Isaacs et al. (1997: 11-15) go further to describe the mechanisms through which informal<br />

communicati<strong>on</strong> achieves its supporting role. The proposed functi<strong>on</strong>s of informal communicati<strong>on</strong> are<br />

(1) tracking people, (2) taking and leaving messages, (3) making meeting arrangements, (4)<br />

delivering documents, (5) giving and getting help and (6) reporting progress and news (Isaacs et al.,<br />

1997: 11-15). Tracking people involves the identifying of the current locati<strong>on</strong> of team members, their<br />

current activities and future plans (Isaacs et al., 1997: 11). Leaving and taking messages is<br />

c<strong>on</strong>cerned with c<strong>on</strong>tacting people through a third pers<strong>on</strong> whereas arranging meetings refers to<br />

making future interacti<strong>on</strong> arrangements with a pers<strong>on</strong> or a group (Isaacs et al., 1997: 12). Document<br />

delivery is the process of handing documents to a recipient with acti<strong>on</strong>s attached such as signing,<br />

proof reading or general perusal (Isaacs et al., 1997: 12-13). Giving and getting help is a joint problem<br />

solving process characterised by questi<strong>on</strong> and answer exchange (Isaacs et al., 1997: 13) while<br />

progress and news reporting is the process of disseminating relevant informati<strong>on</strong> to team members<br />

(Isaacs et al., 1997: 14-15).<br />

Kraut et al. (1990a: 7) suggest that teams need to satisfy three super-tasks in order to be successful,<br />

which are (1) producti<strong>on</strong>, (2) group maintenance and (3) member support. Producti<strong>on</strong> refers to<br />

achieving the goals of the project in which the team is engaged such as writing reports, delivering<br />

presentati<strong>on</strong>s, and all other work needs of the project. Group maintenance is the process of recruiting<br />

and socializing new group members, securing external resources and maintaining group<br />

cohesiveness in order to sustain the team over its lifecycle (Kraut et al., 1990a: 7). Member support is<br />

the process of regulating the feelings of individual group members in order to ensure that they are<br />

satisfied with their work, relati<strong>on</strong>ships and team membership (Kraut et al., 1990a: 7).<br />

The functi<strong>on</strong>s of informal communicati<strong>on</strong> proposed by Isaacs et al. (1997: 11-15) c<strong>on</strong>tribute to the<br />

achievement of the three super-tasks suggested by Kraut et al. (1990a: 7). Help, document delivery<br />

and reporting functi<strong>on</strong>s all aid in the producti<strong>on</strong> needs of the team, while tracking, message taking<br />

and arranging meetings are directly supportive of the group maintenance functi<strong>on</strong> (Isaacs et al., 1997:<br />

14-15). Member support is achieved through the help, tracking and reporting functi<strong>on</strong>s (Isaacs et al.,<br />

1997: 15).<br />

3. Virtual teaming<br />

The internati<strong>on</strong>al business envir<strong>on</strong>ment has become increasingly complex and competitive, forcing<br />

businesses to rethink and modify their operating models (Bharadwaj and Saxena, 2006: 63). These<br />

changes have brought about the creati<strong>on</strong> of virtual teams, defined by Geister et al. (2006: 459 – 460)<br />

as ‘two or more people who work together <strong>on</strong> a mutual goal…interact from different locati<strong>on</strong>s…and<br />

communicate …by means of informati<strong>on</strong> and communicati<strong>on</strong> technology.’ A virtual team can be made<br />

up of a diversely skilled complement of members in order to form the best possible group for a given<br />

project (Pins<strong>on</strong>neault and Caya, 2005: 1). Liu and Burn (2007: 41) identify three typical attributes of<br />

virtual teams as being (1) geographically dispersed, (2) lacking in social c<strong>on</strong>text and (3) lacking in<br />

face-to-face encounters. A fourth dimensi<strong>on</strong> is that virtual teams make significant use of informati<strong>on</strong><br />

and communicati<strong>on</strong> technologies in order to facilitate communicati<strong>on</strong> and coordinati<strong>on</strong> processes and<br />

meet team goals (Maruping and Agarwal, 2004: 975). Maruping and Agarwal (2004: 975) provide an<br />

analysis of virtual communicati<strong>on</strong> capabilities in c<strong>on</strong>trast to the face-to-face medium:<br />

Computer Mediated Communicati<strong>on</strong> (CMC) is superior to face-to-face in brainstorming and<br />

decisi<strong>on</strong> making (Maruping and Agarwal, 2004: 975; Santra and Giri, 2009: 105).<br />

Face-to-face is superior to CMC in c<strong>on</strong>flict management and problem-solving (Maruping and<br />

Agarwal, 2004: 975)<br />

CMC results in lower productivity/performance in virtual teams (Maruping and Agarwal, 2004:<br />

975; Pins<strong>on</strong>neault and Caya, 2005: 8; Liu and Burn, 2007: 45; Siebdrat et al., 2008: 3).<br />

Liu and Burn (2007: 43) offer a model with which to compare satisfacti<strong>on</strong> and performance in face-toface<br />

as well as virtual teams. The model, as illustrated in Figure 1, highlights a distincti<strong>on</strong> made<br />

between virtual and face-to-face teams being that virtual teams are far more task-oriented and use<br />

18


Garth Alistoun and Christopher Upfold<br />

less socio-emoti<strong>on</strong>al processes in order to achieve their outputs (Liu and Burn, 2007: 46). This view is<br />

echoed by Pins<strong>on</strong>neault and Caya (2005: 6) and Maruping and Agarwal (2004: 979) but differs by<br />

suggesting that time impacts virtual team communicati<strong>on</strong> by modifying an initial task focus to a far<br />

more social <strong>on</strong>e as the team develops.<br />

Figure 1: Performance path differences between virtual and face-to-face teams reproduced from Liu<br />

and Burn (2007: 44)<br />

3.1 Trust building<br />

Trust is traditi<strong>on</strong>ally developed by interacting face-to-face and developing a shared social c<strong>on</strong>text<br />

(Kirkman et al., 2002: 69; Pins<strong>on</strong>neault and Caya, 2005: 5; Thomas and Bostrom, 2008; 46).<br />

Research has suggested that computer mediated communicati<strong>on</strong> leads to lower levels of trust<br />

between team members (Geister et al., 2006: 461). Virtual teams, being dispersed by definiti<strong>on</strong>, have<br />

to base their trust <strong>on</strong> something other than social c<strong>on</strong>text. Researchers propose that virtual teams<br />

base their trust <strong>on</strong> task participati<strong>on</strong> and predictable performance of teammates (Kirkman et al., 2002:<br />

69; Pins<strong>on</strong>neault and Caya, 2005: 5; Thomas and Bostrom, 2008; 46). This finding ties in with the Liu<br />

and Burn (2007: 46) model, illustrated in Figure 1 where virtual teams make limited use of socioemoti<strong>on</strong>al<br />

processes in the performance of their goals.<br />

3.2 Informati<strong>on</strong> exchange<br />

As previously menti<strong>on</strong>ed, virtual teams can be made up of a diversely skilled group of people<br />

(Pins<strong>on</strong>neault and Caya, 2005: 1). The diversity of the team means that there exists a large pool of<br />

informati<strong>on</strong> which the team can potentially make use of. Informati<strong>on</strong> exchange has, however, been<br />

dem<strong>on</strong>strated to be surprising poor in virtual teams with <strong>on</strong>e researcher noting that a face-to-face<br />

team under study, exchanged more unique informati<strong>on</strong> in a single meeting than a virtual team<br />

exchanged in three weeks of asynchr<strong>on</strong>ous communicati<strong>on</strong> (Pins<strong>on</strong>neault and Caya, 2005: 7).<br />

3.3 Process gains and losses<br />

According to Kirkman et al. (2002: 71), virtual teams find it more difficult to benefit from process gains<br />

than their face-to-face counterparts. This is because virtual team members rarely, if ever, have the<br />

opportunity to interact informally ‘in a hallway or around a water cooler’ where many of a team’s best<br />

ideas are generated (Kirkman et al., 2002: 71).<br />

19


3.4 Feelings of isolati<strong>on</strong><br />

Garth Alistoun and Christopher Upfold<br />

Virtual teams, as already stated, have little or no opportunity to interact informally/socially (Kirkman et<br />

al., 2002: 72). Without this type of interacti<strong>on</strong> team members feel isolated from <strong>on</strong>e another which<br />

leads to a lower level of work satisfacti<strong>on</strong> and performance. This reducti<strong>on</strong> in relati<strong>on</strong>ship building<br />

opportunities is illustrated in Figure 1 where virtual teams base performance and satisfacti<strong>on</strong><br />

outcomes <strong>on</strong> task processes al<strong>on</strong>e.<br />

3.5 Participati<strong>on</strong><br />

While it was previously suggested that participati<strong>on</strong> in virtual teams was assumed to be more equal<br />

than in co-located teams, Pins<strong>on</strong>neault and Caya (2005: 7) suggest there is no evidence supporting<br />

this assumpti<strong>on</strong>. They also suggest that percepti<strong>on</strong>s of fairness in labour divisi<strong>on</strong> were found to be<br />

lower in studied virtual teams than in their face-to-face counterparts.<br />

3.6 Coordinati<strong>on</strong><br />

Virtual teams cannot rely <strong>on</strong> physical coordinati<strong>on</strong> activities such as direct supervisi<strong>on</strong> and proximity<br />

of participants (Pins<strong>on</strong>neault and Caya, 2005: 8). Hence, coordinati<strong>on</strong> has been dem<strong>on</strong>strated to be<br />

more difficult in a virtual envir<strong>on</strong>ment than a co-located <strong>on</strong>e. They also argued that coordinati<strong>on</strong><br />

efficiency is lower in virtual teams and decreases over time (Pins<strong>on</strong>neault and Caya, 2005: 8). Finally,<br />

these authors suggest that while coordinati<strong>on</strong> interventi<strong>on</strong>s designed to manage c<strong>on</strong>tent and timing of<br />

intra-team communicati<strong>on</strong> have been dem<strong>on</strong>strated to alleviate the decrease in coordinati<strong>on</strong><br />

efficiency, they have negatively impacted <strong>on</strong> team trust (Pins<strong>on</strong>neault and Caya, 2005: 8).<br />

3.7 Cohesi<strong>on</strong><br />

Cohesi<strong>on</strong> has been shown to be positively associated with team performance in both virtual and<br />

physical teams (Pins<strong>on</strong>neault and Caya, 2005: 6). Virtual teams have to communicate far more<br />

frequently than face-to-face teams in order to achieve the same level of team cohesiveness. This<br />

situati<strong>on</strong> is alleviated over time as team members become familiar with communicati<strong>on</strong> technologies<br />

and are able to exchange more pers<strong>on</strong>al informati<strong>on</strong> with greater effectiveness (Pins<strong>on</strong>neault and<br />

Caya, 2005: 6).<br />

4. Functi<strong>on</strong>al system requirements<br />

Informal communicati<strong>on</strong> plays an important role in supporting effective teamwork through workexecuti<strong>on</strong>,<br />

group co-ordinati<strong>on</strong>, socializati<strong>on</strong> and team building processes (Whittaker et al., 1994:<br />

131). Restricti<strong>on</strong> of these communicative processes has a negative impact <strong>on</strong> task-performance and<br />

team satisfacti<strong>on</strong> as evidenced in virtual collaborati<strong>on</strong> (Isaacs et al., 1997: 9). In order to overcome<br />

the challenges faced by different virtual teams, a communicati<strong>on</strong> soluti<strong>on</strong> will have certain<br />

requirements placed up<strong>on</strong> it to be effective.<br />

In an attempt to identify and explore some of the requirements believed to be necessary for a<br />

potential virtual team communicati<strong>on</strong> soluti<strong>on</strong>, a system called the VideoWindow system was<br />

implemented at Bellcore and studied by Fish et al. (1990).<br />

Figure 2: Image of the VideoWindow System (Fish et al. 1990)<br />

20


Garth Alistoun and Christopher Upfold<br />

The system c<strong>on</strong>sisted of two large screens mounted <strong>on</strong> the wall of two research lounges <strong>on</strong> different<br />

floors of a building, linked by video recorders and directi<strong>on</strong>al microph<strong>on</strong>es. Establishing a c<strong>on</strong>necti<strong>on</strong><br />

was meant to be as simple as looking at the screen for an available partner and initiating a<br />

c<strong>on</strong>versati<strong>on</strong> with that partner. In order to ensure co-presence of potential c<strong>on</strong>versati<strong>on</strong>al partners,<br />

free coffee was provided at each end of the system for the durati<strong>on</strong> of the experiment.<br />

The results of the Fish et al. (1990: 6-10) experiments are summarised under seven headings:<br />

Transparency – Defined by Fish et al. (1990: 6) as the ability to distinguish between<br />

c<strong>on</strong>versati<strong>on</strong>s using the system and those that occur face-to-face. The VideoWindow was found<br />

to alter c<strong>on</strong>versati<strong>on</strong>s in minor ways where users tended to speak relatively louder and often<br />

embedded comments <strong>on</strong> the system itself in their interacti<strong>on</strong>s. The most important finding under<br />

this heading was that c<strong>on</strong>versati<strong>on</strong> opportunities over the VideoWindow had a substantially lower<br />

c<strong>on</strong>versi<strong>on</strong> rate than face-to-face opportunities, 17% in comparis<strong>on</strong> to 41% (Fish et al., 1990: 7).<br />

Reciprocity – Defined by the Oxford English Dicti<strong>on</strong>ary as ‘a state or relati<strong>on</strong>ship in which there<br />

is mutual acti<strong>on</strong>, influence, giving and taking, corresp<strong>on</strong>dence, etc., between two parties’<br />

(University of Chicago, 2002). The VideoWindow system was found to have certain reciprocity<br />

problems associated with it (Fish et al., 1990: 7). According to Fish et al. (1990: 7), the principle in<br />

face-to-face c<strong>on</strong>versati<strong>on</strong> opportunities is that if you can see some<strong>on</strong>e, then they can see you and<br />

if you can hear some<strong>on</strong>e, then they can hear you. This principle was not preserved in the<br />

VideoWindow system where both the camera and the microph<strong>on</strong>es had a limited range of<br />

functi<strong>on</strong>ality which resulted in a failure to c<strong>on</strong>vert many c<strong>on</strong>versati<strong>on</strong> opportunities into<br />

c<strong>on</strong>versati<strong>on</strong>s.<br />

Privacy – The VideoWindow system did not support the ability to make a c<strong>on</strong>versati<strong>on</strong> private.<br />

This was <strong>on</strong>e of the most requested features by the study group (Fish et al., 1990: 8).<br />

Architectural and Envir<strong>on</strong>mental C<strong>on</strong>text – The VideoWindow system is publicly situated in<br />

order to allow for sp<strong>on</strong>taneous interacti<strong>on</strong>s mimicking hallway or coffee room run-ins between<br />

colleagues. This imposes a behavioural cost <strong>on</strong> users if they have no reas<strong>on</strong> to be in the public<br />

area when they wish to use the system (Fish et al., 1990: 8).<br />

Social and Organizati<strong>on</strong>al C<strong>on</strong>text - Fish et al., (1990: 8) propose that the VideoWindow<br />

system would <strong>on</strong>ly be effective if it links users who already know <strong>on</strong>e another or have an explicit<br />

reas<strong>on</strong> to communicate.<br />

C<strong>on</strong>versati<strong>on</strong>al Regulati<strong>on</strong> – The initiati<strong>on</strong> and maintenance of a face-to-face c<strong>on</strong>versati<strong>on</strong> is<br />

regulated by a complex set of mechanisms (Fish et al., 1990: 8). A technical failing of the<br />

VideoWindow system due to the placement of cameras is that it does not support eye c<strong>on</strong>tact<br />

which is suggested to be an important mechanism for regulating social interacti<strong>on</strong>.<br />

Social Relati<strong>on</strong>ships – Fish et al. (1990: 9) hypothesised that the VideoWindow system would<br />

impact the relati<strong>on</strong>ships between the distributed participants over time. This would lead to positive<br />

effects of proximity such as greater familiarity and pers<strong>on</strong> liking (Fish et al., 1990: 9; Kraut et al.,<br />

1990a: 23-24).<br />

As already suggested, informal communicati<strong>on</strong> has certain functi<strong>on</strong>al requirements that must be met<br />

in order to positively impact virtual team collaborati<strong>on</strong> (Maruping and Agarwal, 2004: 976). From the<br />

earlier definiti<strong>on</strong> of informal communicati<strong>on</strong>, informal communicati<strong>on</strong> is unscheduled, interactive and<br />

rich between random, out of role participants (Kraut et al., 1990a: 5). The following are proposed to be<br />

the requirements of a virtual informal communicati<strong>on</strong> system:<br />

4.1 Co-presence<br />

Isaacs et al. (1997: 10) support a finding of Kraut and Streeter (1995) that sp<strong>on</strong>taneous interacti<strong>on</strong> is<br />

under-utilised relative to its value, whereas formal interacti<strong>on</strong> is over-used. This finding is supported<br />

by many researchers who indicate the lack of support for serendipitous communicati<strong>on</strong> in mainstream<br />

communicati<strong>on</strong> technologies (v<strong>on</strong> Bismarck et al., 1999: 5-6; Kraut et al., 1990a: 4; Whittaker et al.,<br />

1994: 131). The definiti<strong>on</strong> of informal interacti<strong>on</strong> indicates that an effective informal communicati<strong>on</strong><br />

technology should allow for sp<strong>on</strong>taneous interacti<strong>on</strong> by creating co-presence between possible<br />

c<strong>on</strong>versati<strong>on</strong>al participants (Isaacs et al., 1997: 21). Co-presence is defined by Kraut et al. (1990a:<br />

33) as the mechanism by which possible c<strong>on</strong>versati<strong>on</strong> participants are brought together and are made<br />

aware of <strong>on</strong>e another’s availability. Kraut et al. (1990a: 33) posit that the essence of computermediated<br />

communicati<strong>on</strong> is co-presence without physical proximity. Co-presence is not simply a<br />

21


Garth Alistoun and Christopher Upfold<br />

matter of randomly c<strong>on</strong>necting participants but also providing a communicati<strong>on</strong> c<strong>on</strong>text or ‘comm<strong>on</strong><br />

ground’ in which the possibility of communicati<strong>on</strong> is created (Nardi, 2005: 91-92). The possibility of<br />

virtual informal communicati<strong>on</strong> is created by creating an envir<strong>on</strong>ment and a social c<strong>on</strong>text in which<br />

users can sp<strong>on</strong>taneously interact (Isaacs et al., 1997: 21). The c<strong>on</strong>text of c<strong>on</strong>versati<strong>on</strong> may take the<br />

form of a coffee room, a lounge, a work area or any other publicly available space available to users<br />

(Whittaker et al. 1994: 131). The social c<strong>on</strong>text created will help to reduce informati<strong>on</strong> exchange<br />

challenges (Nardi, 2005: 92) by providing a c<strong>on</strong>text for trust creati<strong>on</strong> (Kirkman et al., 2002: 69;<br />

Pins<strong>on</strong>neault and Caya, 2005: 5; Thomas and Bostrom, 2008; 46). An alternate view is presented by<br />

Dourish et al. (1996: 34) who state that face-to-face communicati<strong>on</strong> in the real world should not be<br />

used as a baseline for evaluating virtual communicati<strong>on</strong> systems. They propose, echoing the views of<br />

Maruping and Agarwal (2002), that teams have different communicative requirements at different<br />

stages of their development (Dourish et al., 1996: 34). As teams become more used to interacting<br />

with <strong>on</strong>e another and increasingly familiar with the technology they use to interact, they develop a<br />

new communicative behaviour that cannot be compared to face-to-face interacti<strong>on</strong> (Dourish et al.,<br />

1996: 34). Dourish et al. (1996: 34) propose that a team will evolve with a communicati<strong>on</strong> technology<br />

to use it effectively and achieve the same performance and satisfacti<strong>on</strong> levels as co-located teams.<br />

This view is c<strong>on</strong>firmed by Pins<strong>on</strong>neault and Caya (2005: 8-9), however, they do identify other<br />

challenges which are not necessarily alleviated by time and experience (Pins<strong>on</strong>neault and Caya,<br />

2005: 10).<br />

4.2 C<strong>on</strong>centrati<strong>on</strong> of suitable partners<br />

As already discussed, in placing the system into a social and organisati<strong>on</strong> c<strong>on</strong>text it would <strong>on</strong>ly be<br />

effective if it links users who already know <strong>on</strong>e another or have an explicit reas<strong>on</strong> to communicate<br />

(Fish et al., 1990: 8). In co-locati<strong>on</strong>, this is achieved by placing people who need to work together or<br />

share a comm<strong>on</strong> interest close to <strong>on</strong>e another (Kraut et al., 1990a: 33). This is encapsulated in the<br />

system’s architectural and envir<strong>on</strong>mental as well as social and organizati<strong>on</strong>al c<strong>on</strong>texts.<br />

4.3 Low behavioural cost<br />

Kraut et al. (1990a: 33) define behavioural cost as ‘the amount of effort needed to initiate and c<strong>on</strong>duct<br />

a c<strong>on</strong>versati<strong>on</strong>’. The perceived behavioural cost of a communicati<strong>on</strong> system is proposed to be an<br />

important determinant of its usefulness (Kraut et al., 1990a: 34). A pers<strong>on</strong> will not use a<br />

communicati<strong>on</strong> device if they believe that the behavioural cost is too high (Kraut et al., 1990a: 34).<br />

Making c<strong>on</strong>tact with another pers<strong>on</strong> is often a by-product of another activity and a communicati<strong>on</strong><br />

system which aims to support sp<strong>on</strong>taneous informal encounters should embody this fact (Kraut et al.,<br />

1990a: 33).<br />

4.4 Visual channel<br />

A hypothesis made by Isaacs et al. (1997: 23) is that c<strong>on</strong>versati<strong>on</strong>al partners will make use of video<br />

data to establish whether a user is open to interacti<strong>on</strong>. However, both Whittaker et al. (1994: 135) and<br />

Kraut et al. (1990a: 33) agree that frequent interactors will initiate a c<strong>on</strong>versati<strong>on</strong> regardless of the<br />

receivers apparent openness to interacti<strong>on</strong>. Users simply make use of the system to see whether the<br />

receiver is available rather than ready to c<strong>on</strong>verse (Whittaker et al., 1994: 135; Kraut et al., 1990a:<br />

33). The visual channel therefore plays an important role as a stimulus for c<strong>on</strong>versati<strong>on</strong> (Kraut et al.,<br />

1990a: 34).<br />

4.5 Document sharing<br />

Documents play an important role in initiating, re-initiating and sustaining c<strong>on</strong>versati<strong>on</strong>s (Whittaker et<br />

al., 1994: 135). An effective informal communicati<strong>on</strong> system should employ some form of document<br />

sharing and manipulati<strong>on</strong> (Whittaker et al., 1994: 135).<br />

4.6 Multiple complementary systems<br />

Both Isaacs et al. (1997: 25) and Pins<strong>on</strong>neault and Caya (2005: 11) propose that no <strong>on</strong>e system can<br />

support all of the functi<strong>on</strong>al needs of informal communicati<strong>on</strong>. Virtual teams that make use multiple<br />

communicati<strong>on</strong> systems have shown a greater level of satisfacti<strong>on</strong>, equality of participati<strong>on</strong> and<br />

quality of outputs than those using a single medium (Pins<strong>on</strong>neault and Caya, 2005: 11). They<br />

propose that multiple systems will have the flexibility that a virtual team requires in order to exchange<br />

and process informati<strong>on</strong> effectively.<br />

22


Garth Alistoun and Christopher Upfold<br />

Capturing the role of informal communicati<strong>on</strong>, the identified dysfuncti<strong>on</strong>s of virtual teams and the<br />

acknowledged functi<strong>on</strong>al system requirements, the proposed framework of an effective informal<br />

communicati<strong>on</strong> system is illustrated in Figure 3.<br />

Figure 3: A proposed framework of an effective informal communicati<strong>on</strong> system<br />

5. C<strong>on</strong>clusi<strong>on</strong><br />

Existing research suggests that virtual teaming technologies do not take into account the informal<br />

communicati<strong>on</strong> needs of teams. Informal communicati<strong>on</strong> has been shown to play an important role in<br />

the producti<strong>on</strong>, group maintenance and member support goals of teams. Virtual teaming is pr<strong>on</strong>e to a<br />

c<strong>on</strong>siderable list of challenges due to the dispersed nature of the team. In the course of the literature<br />

review, these challenges were described as (1) trust building, (2) informati<strong>on</strong> exchange, (3) process<br />

gains and losses, (4) feelings of isolati<strong>on</strong>, (5) participati<strong>on</strong>, (6) coordinati<strong>on</strong> and (7) cohesi<strong>on</strong>. In order<br />

to limit the impact of the identified challenges and help virtual teams meet their producti<strong>on</strong>, group<br />

maintenance and member support needs, five functi<strong>on</strong>al needs of an effective informal<br />

communicati<strong>on</strong> system are distilled from the literature. The identified needs are given as (1) copresence,<br />

(2) low behavioural cost, (3) visual channel, (4) document sharing and (5) multiple<br />

complimentary systems.<br />

The proposed framework can be used to assess existing and in-development virtual communicati<strong>on</strong><br />

systems in terms of their likely impact <strong>on</strong> virtual team effectiveness.<br />

6. Future work<br />

Given that this proposed framework is purely theoretical in nature, further research will be required to<br />

develop the framework further. Masters students in the Department of Informati<strong>on</strong> Systems, Rhodes<br />

University have c<strong>on</strong>structed a working VirtualWindow system. The proposed framework will be used<br />

to guide the evaluati<strong>on</strong> of this VirtualWindow system together with other existing virtual<br />

communicati<strong>on</strong> systems. In particular, a case study has been planned and will be c<strong>on</strong>ducted between<br />

two remotely located project teams working within the IT Systems Development field in South Africa.<br />

23


Garth Alistoun and Christopher Upfold<br />

The intenti<strong>on</strong> is to evaluate the custom built system, described above, in an attempt to evaluate to<br />

what extent the system addresses the five functi<strong>on</strong>al needs c<strong>on</strong>sidered necessary for implementing<br />

an effective informal communicati<strong>on</strong> system. Based <strong>on</strong> the outcome of the case study, the Virtual<br />

Window system will be updated in an attempt to address shortcomings and the framework will also be<br />

reflected <strong>on</strong> and refined based <strong>on</strong> the findings of the experiment. The important goal of this future<br />

work is to introduce and evaluate a highly effective system for facilitating informal communicati<strong>on</strong><br />

within virtual teams.<br />

References<br />

Bharadwaj, S. S. and Saxena, K. B. C. (2006) “Impacting the Processes of Global Software Teams: A<br />

Communicati<strong>on</strong> Technology Perspective”. VISION - The Journal of Business Perspective. 10(4): 63-75.<br />

Dourish, P., Adler, A., Belotti, V and Henders<strong>on</strong>, A. (1996) “Your Place or Mine? Learning from L<strong>on</strong>g-Term Use of<br />

Audio-Video Communicati<strong>on</strong>”. Computer Supported Cooperative Work. 5(1): 33-62.<br />

Fish, R. S., Kraut, R. E. and Chalf<strong>on</strong>te B. L. (1990) “The VideoWindow System in Informal Communicati<strong>on</strong>s”.<br />

Proceedings of the 1990 ACM c<strong>on</strong>ference <strong>on</strong> computer supported cooperative work.<br />

Geister, S., K<strong>on</strong>radt, U. and Hertel, G. (2006) “Effects of Process Feedback <strong>on</strong> Motivati<strong>on</strong>, Satisfacti<strong>on</strong> and<br />

Performance in Virtual Teams”. Small Group Research. 37(5): 459-489.<br />

Isaacs, E. A., Whittaker, S., Frohlich, D. and O’C<strong>on</strong>aill, B. (1997) “Informal Communicati<strong>on</strong> re-examined: New<br />

functi<strong>on</strong>s for video in supporting opportunistic encounters”. In: Finn, K., Sellen, A. and Wilbur, S. (eds).<br />

Video-Mediated Communicati<strong>on</strong>. Mahwah, New Jersey: Lawrence Ehrlbaum.<br />

Kirkman, B. L., Rosen, B., Gibs<strong>on</strong>, C. B., Tesluk, P. E. and McPhers<strong>on</strong>, S. O. (2002) “Five Challenges to Virtual<br />

Team Success: Less<strong>on</strong>s from Sabre Inc”. The Academy of <strong>Management</strong> Executive. 16(3): 67-79<br />

Liu, Y. C. and Burn, J. M. (2007) “Improving Value Returns from Virtual Teams”. Proceedings of the <str<strong>on</strong>g>European</str<strong>on</strong>g><br />

and Mediterranean <str<strong>on</strong>g>C<strong>on</strong>ference</str<strong>on</strong>g> <strong>on</strong> Informati<strong>on</strong> Systems.<br />

Maruping, l. M. and Agarwal, R. (2004) “Managing Team Interpers<strong>on</strong>al Processes through Technology: A Task–<br />

Technology Fit Perspective”. Journal of Applied Psychology. 89(6): 975-990<br />

Nardi, B. A. (2005) “Bey<strong>on</strong>d Bandwidth: Dimensi<strong>on</strong>s of C<strong>on</strong>necti<strong>on</strong> in Interpers<strong>on</strong>al Communicati<strong>on</strong>”. Computer<br />

Supported Cooperative Work. 14(2): 91-130.<br />

Pins<strong>on</strong>neault, A. and Caya, O. (2005) “Virtual Teams: What We Know and What We D<strong>on</strong>’t Know”. Internati<strong>on</strong>al<br />

Journal of e-Collaborati<strong>on</strong>. 1(3): 1-16.<br />

Santra, T. and Giri, V. N. (2009) “Analyzing Computer-Mediated Communicati<strong>on</strong> and Organizati<strong>on</strong>al<br />

Effectiveness”. The Review of Communicati<strong>on</strong>. 9(1): 100-109.<br />

Siebdrat, F., Hoegl, M. and Ernst, H. (2008) “The Bright Side of Virtual Collaborati<strong>on</strong>: How Teams Can Profit from<br />

Dispersi<strong>on</strong>”. Proceedings of the 2008 Academy of <strong>Management</strong> Best Paper Awards.<br />

Thomas, D. and Bostrom, R. (2008) “Building Trust and Cooperati<strong>on</strong> Through Technology Adapti<strong>on</strong> in Virtual<br />

Teams: Empirical Field Evidence”. Informati<strong>on</strong> Systems <strong>Management</strong>. 25(1): 45-56.<br />

University of Chicago (2002) Reciprocity. [<strong>on</strong>line]. Available at:<br />

http://csmt.uchicago.edu/glossary2004/reciprocity.htm. [Accessed 7 May 2010]<br />

V<strong>on</strong> Bismarck, W.-B., Bungard, W, and Held, M. (1999) “Is informal communicati<strong>on</strong> needed, wanted and<br />

supported?” Proceedings of the 8 th internati<strong>on</strong>al c<strong>on</strong>ference <strong>on</strong> human computer interacti<strong>on</strong>.<br />

Whittaker, S., Frohlich, D. and Daly-J<strong>on</strong>es, O. (1994) “Informal Workplace Communicati<strong>on</strong>: What it is like and<br />

how might we support it?” Proceedings of the SIGCHI c<strong>on</strong>ference <strong>on</strong> human factors in computing systems:<br />

celebrating interdependence.<br />

24


IS C<strong>on</strong>sultants and SMEs: A Competence Perspective<br />

Adrian Bradshaw, Paul Cragg and Venkat Pulakanam<br />

College of Business and Ec<strong>on</strong>omics, Canterbury University, Christchurch,<br />

New Zealand<br />

paul.cragg@canterbury.ac.nz<br />

Abstract: Many small and medium-sized enterprises (SMEs) lack in-house resources, including IS knowledge<br />

and skills. As a result, they often turn to c<strong>on</strong>sultants for assistance with IS projects, eg, helping the firm select<br />

and implement a new system. While there is some evidence that this practice increases IS success, there has<br />

been no attempt to discover if SMEs gain anything other than a new system from such projects. Thus this study<br />

aimed to determine what SMEs gain from engaging IS c<strong>on</strong>sultants. In particular, the study aimed to identify any<br />

improvements to internal IS competences. In brief, did the c<strong>on</strong>sultant engagement help improve IS skills and<br />

abilities within the SME? A multiple case study approach was adopted. Data was collected from SMEs that had<br />

engaged c<strong>on</strong>sultants to implement an accounting informati<strong>on</strong> system. In additi<strong>on</strong>, to provide a broader<br />

perspective, interviews were c<strong>on</strong>ducted with c<strong>on</strong>sultants who specialise in assisting SMEs with IS. Resourcebased<br />

theory was used as a lens to help analyse the case evidence. Each case was examined to identify<br />

instances where c<strong>on</strong>sultants influenced a competence, ie, had an impact <strong>on</strong> the creati<strong>on</strong> or use of any<br />

competences. The cases provide evidence that SMEs lack many IS abilities. The findings indicate that<br />

c<strong>on</strong>sultants compensate for a lack of IS competences rather than help build in-house competences. C<strong>on</strong>sultants<br />

help many SMEs overcome their lack of IS competences. The study adds to our understanding of c<strong>on</strong>sultants<br />

acting as intermediaries to assist and advise firms. In this intermediary role as ‘c<strong>on</strong>duits’, c<strong>on</strong>sultants provide<br />

advice to assist many SMEs by finding appropriate products, implementing the system, integrating software with<br />

existing systems, and training and support. C<strong>on</strong>sultant attributes, eg, technical, soft skills and training skills,<br />

influence project success. The study provides new insights into a particularly significant relati<strong>on</strong>ship for SMEs<br />

and thus provides a step towards improving our understanding of IS success in SMEs.<br />

Keywords: IS projects, IS c<strong>on</strong>sultants, SMEs, competences<br />

1. Introducti<strong>on</strong><br />

With relatively low levels of IS expertise, the typical small and medium-sized enterprise (SME) turns to<br />

external experts when faced with a major IS project, for example, when acquiring a new system. Prior<br />

studies have shown that external expertise is a major predictor of IS success for SMEs (Th<strong>on</strong>g, 2001;<br />

Th<strong>on</strong>g et al., 1997; de Guinea et al., 2005; Bruque and Moyano, 2007). However, despite its<br />

importance to IS success, relatively little research has focused <strong>on</strong> the relati<strong>on</strong>ship between SMEs and<br />

IS external experts. This study helps address this gap by examining major IS projects in SMEs to<br />

explore the role of IS experts, how external experts add value, and how external experts influence<br />

project success.<br />

A multiple case study approach was adopted involving both SMEs and IS c<strong>on</strong>sultants. The study<br />

focused <strong>on</strong> IS projects where new accounting informati<strong>on</strong> systems (AIS) had been installed, including<br />

a new versi<strong>on</strong> of an existing system. The evidence was used to examine the role played by IS<br />

c<strong>on</strong>sultants in implementing IS projects for SMEs. SMEs were defined for this study as independent<br />

firms with between 5 and 50 employees. A c<strong>on</strong>sultant was c<strong>on</strong>sidered as any pers<strong>on</strong> or organisati<strong>on</strong><br />

that is certified to install or implement an accounting informati<strong>on</strong> system. This included accounting<br />

and c<strong>on</strong>sulting firms.<br />

2. Prior literature<br />

There is evidence that SMEs engage c<strong>on</strong>sultants for various tasks, including selecting and<br />

implementing packaged software (Howcroft and Light, 2008), software and Web applicati<strong>on</strong><br />

development, project management, and benchmarking (Nevo et al., 2007). Furthermore, studies have<br />

identified the importance of external experts like c<strong>on</strong>sultants in aiding SMEs with IS projects (Soh,<br />

Yap, & Raman, 1992; Th<strong>on</strong>g, Yap, & Raman, 1994; Th<strong>on</strong>g, Yap, & Raman, 1997; Th<strong>on</strong>g, 2001; de<br />

Guinea et al., 2005).<br />

The broader IS literature indicates the following four reas<strong>on</strong>s why firms in general engage<br />

c<strong>on</strong>sultants:<br />

Firms may engage c<strong>on</strong>sultants for their knowledge and expertise since firms may not have<br />

sufficient knowledge or expertise in-house (Nevo et al., 2007)<br />

Firms may engage c<strong>on</strong>sultants as an alternative to hard to find IS staff (Nevo et al., 2007)<br />

25


Adrian Bradshaw, Paul Cragg and Venkat Pulakanam<br />

Firms engage c<strong>on</strong>sultants for knowledge transfer to internal IS staff and to gain technical knowhow<br />

(Nevo et al., 2007).<br />

Firms may hire c<strong>on</strong>sultants to compensate for a lack of capability. The lack of these capabilities<br />

represents a barrier to technology transfer, especially in smaller and less experienced firms.<br />

C<strong>on</strong>sultants act as intermediaries to assist and advise firms, effectively compensating for a lack of<br />

capability (Bessant and Rush, 1995).<br />

Champi<strong>on</strong> et al. (1990) and Basil et al. (1997) sort the roles generally played by c<strong>on</strong>sultants into nine<br />

categories: hands-<strong>on</strong> expert, modeller, partner, coach, teacher or trainer, technician, counsellor,<br />

facilitator, and reflective observer. Bessant and Rush (1995) highlight four roles of c<strong>on</strong>sultants. Firstly,<br />

the traditi<strong>on</strong>al role of c<strong>on</strong>sulting sees c<strong>on</strong>sultants transferring specialised, expert knowledge to clients.<br />

Sec<strong>on</strong>dly, c<strong>on</strong>sultants engage in the role of experience sharing, either implicitly or explicitly. Thirdly,<br />

c<strong>on</strong>sultants act as ‘marriage brokers’, where c<strong>on</strong>sultants are a single point of c<strong>on</strong>tact for the client to<br />

access a wide range of specialist services. The fourth role is a diagnostic role. In this role c<strong>on</strong>sultants<br />

help their clients articulate and define their needs.<br />

There is also evidence that c<strong>on</strong>sultants play various roles in SMEs. Traditi<strong>on</strong>ally in IS research, the<br />

main role of the IS c<strong>on</strong>sultant in SMEs has been expressed as a mediator role. Th<strong>on</strong>g (2001) and de<br />

Guinea et al. (2005) note that in small businesses, c<strong>on</strong>sultants act as mediators for the lack of IS<br />

skills and lack of expertise. The role of c<strong>on</strong>sultants has also been seen as intermediary. C<strong>on</strong>sultants<br />

act as bridging intermediaries by disseminating knowledge (Carey, 2008). C<strong>on</strong>sultants also act as<br />

‘c<strong>on</strong>duits’ by standing between IS suppliers and SMEs (Howcroft and Light, 2008). In these<br />

intermediary roles, c<strong>on</strong>sultants carry out several activities and services. Howcroft and Light (2008)<br />

point out that c<strong>on</strong>sultants provide services such as advice to assist with finding appropriate software,<br />

the implementati<strong>on</strong> and customisati<strong>on</strong> of the software, training and support service and the integrati<strong>on</strong><br />

of software with existing systems. In the bridging role c<strong>on</strong>sultants carry out activities such as:<br />

transferring specialised knowledge; sharing ideas and experiences; acting as a point of c<strong>on</strong>tact for a<br />

wide range of specialised services; and assisting clients to clearly specify their particular needs<br />

(Carey, 2008).<br />

Prior research has used the resource-based view to examine IS in SMEs (Th<strong>on</strong>g, 2003; Caldiera and<br />

Ward, 2003; Eikebrokk and Olsen, 2007; Butler and Murphy, 2008; and Cragg et al., 2011). Some of<br />

this has identified important IS competences, ie, skills and abilities, that are applicable to IS in SMEs.<br />

For example, Th<strong>on</strong>g (2001) developed a resource-based model of IS implementati<strong>on</strong> in small firms. It<br />

was shown that small firms with successful IS had highly effective external experts, adequate IS<br />

investment, high users’ IS knowledge, high user involvement, and high CEO support. External<br />

expertise was found to be a predominant key factor of IS implementati<strong>on</strong> success in small<br />

businesses. Butler and Murphy (2008) used the resource-based view to understand how small to<br />

medium software enterprises (SMSEs) build and apply business and IS capabilities. Their findings<br />

indicated that managing external relati<strong>on</strong>ships was a core business and IS capability, whatever the<br />

period of an SMSE’s evoluti<strong>on</strong>. Caldiera and Ward (2003) show the great importance of IS<br />

knowledge, ie, either within the firm or in a closely associated specialist enterprise, like a c<strong>on</strong>sultant.<br />

This suggests that for SMEs, c<strong>on</strong>sultants play a vital role by supplying IS knowledge.<br />

Scupola (2008) identified three important competences at the managerial level for SMEs:<br />

Visi<strong>on</strong> - understanding how the system could add value to the company and c<strong>on</strong>tribute to the<br />

company’s business strategy.<br />

Value - finding out what value the system could bring to the company.<br />

C<strong>on</strong>trol - ways/initiatives to encourage and enforce assimilati<strong>on</strong> of the system at the individual<br />

level.<br />

Scupola (2008) also suggested three competences that are key at the individual level: technical skills,<br />

interpers<strong>on</strong>al skills and c<strong>on</strong>ceptual skills. Two other frameworks also identify IS competences specific<br />

to SMEs. Eikebrokk and Olsen (2007) provide a total of seven competences associated with<br />

developing e-business in SMEs. Cragg et al.’s (2011) framework identifies six macro-competences,<br />

which encompass a total of twenty-two competences, including the ability to, eg, define IS<br />

requirements, access IS knowledge, manage change, and project management. The c<strong>on</strong>tent of all<br />

three frameworks is summarised in Table 1.<br />

26


Adrian Bradshaw, Paul Cragg and Venkat Pulakanam<br />

Table 1: IS competences from the SME literature<br />

Eikebrokk & Olsen (2007) Scupola (2008) Cragg et al (2011)<br />

C<strong>on</strong>cept of e-business<br />

Strategic Planning<br />

IT-business process<br />

IT management<br />

Systems and infrastructure<br />

SourcingAlignment<br />

Visi<strong>on</strong><br />

Value<br />

C<strong>on</strong>trol<br />

Technical skills<br />

Interpers<strong>on</strong>al skills<br />

C<strong>on</strong>ceptual skills<br />

Business and IS strategic thinking<br />

Define IS c<strong>on</strong>tributi<strong>on</strong><br />

Define the IS strategy<br />

Exploitati<strong>on</strong><br />

Deliver soluti<strong>on</strong>s<br />

Supply<br />

In summary, research <strong>on</strong> the impact of c<strong>on</strong>sultants <strong>on</strong> SMEs is underdeveloped. Although earlier<br />

studies indicated that external experts have a significant influence <strong>on</strong> IS success, subsequent studies<br />

provide little understanding of their influence. Prior research provides few insights into how external<br />

experts influence IS projects and whether they influence in-house competences.<br />

3. Research objectives and methods<br />

The study aimed to gain a deeper understanding of the role played by IS c<strong>on</strong>sultants and their impact<br />

<strong>on</strong> SMEs. A two-phase approach was adopted. Phase 1 was exploratory as there was relatively little<br />

prior research examining the interplay between IS c<strong>on</strong>sultants and SMEs. The aim for phase 1 was<br />

to:<br />

Determine why SMEs engaged c<strong>on</strong>sultants.<br />

Identify what tasks were undertaken by c<strong>on</strong>sultants.<br />

The sec<strong>on</strong>d phase was designed to built <strong>on</strong> phase 1, with a focus <strong>on</strong> IS competences. The aim of<br />

phase 2 was to:<br />

Determine if c<strong>on</strong>sultants influence IS competences in SMEs.<br />

As phase 1 was exploratory, a multiple case study design was deemed appropriate, based <strong>on</strong><br />

Eisenhardt (1989) and Yin (2009). Phase 1 commenced with developing a case study protocol to<br />

serve as an interview guide for data collecti<strong>on</strong>. The protocol explored informati<strong>on</strong> relating to the<br />

background of the participants, the c<strong>on</strong>sultant engagement process, the role that c<strong>on</strong>sultants play, the<br />

effectiveness of c<strong>on</strong>sultants and the success of the project. The initial protocol was improved through<br />

the use of a pilot case study. The study was c<strong>on</strong>ducted in New Zealand. The multiple-case study<br />

design c<strong>on</strong>sisted of four SMEs and three c<strong>on</strong>sultants. The SMEs involved met the following criteria:<br />

Implemented an AIS within the last 3 years<br />

Had 5 to 50 employees<br />

Used a c<strong>on</strong>sultant to assist with the implementati<strong>on</strong><br />

Tables 2 and 3 provide a summary of the SMEs and c<strong>on</strong>sultants involved in the study. Face-to-face<br />

interviews were used as the major method of collecting data. For the SMEs, the interviews involved<br />

<strong>on</strong>e or more senior managers. All interviews were recorded and transcribed. In additi<strong>on</strong>, brochures<br />

and website were used to gather supporting material. The data were ported into Nvivo and coded and<br />

analysed using several techniques inherent within the software, including memo writing, annotating,<br />

searching, pattern matching and modelling.<br />

Table 2: Summary of the backgrounds of the four SMEs interviewed.<br />

Case<br />

Number of<br />

Staff<br />

Sector AIS Type of C<strong>on</strong>sultant<br />

SP 30 Manufacturing Accredo<br />

Independentreseller<br />

Project<br />

Outcome<br />

Main Interviewee<br />

Successful Managing Director<br />

AB 17 Services MYOB Accountant Successful Practice Manager<br />

AR 16 Manufacturing Infusi<strong>on</strong><br />

Independentreseller<br />

Not successful<br />

Financial<br />

C<strong>on</strong>troller<br />

DL 11 Manufacturing QuickBooks Accountant Successful Owner-Manager<br />

Table 3: Summary of the backgrounds of the three c<strong>on</strong>sultants interviewed<br />

C<strong>on</strong>sultant Number of employees Type of C<strong>on</strong>sultant AIS<br />

AI 5 Independent-reseller Accredo<br />

ER 5 Independent-reseller Accredo<br />

AT 23 Accountant MYOB<br />

27


4. Findings<br />

Adrian Bradshaw, Paul Cragg and Venkat Pulakanam<br />

The cases revealed a range of reas<strong>on</strong>s why the firms engaged c<strong>on</strong>sultants. The main reas<strong>on</strong>s were a<br />

lack of IS knowledge, a lack of IS skills, and a lack of accounting knowledge and accounting skill. For<br />

example, here are some paraphrased comments from the interviews with SME managers:<br />

we did not really understand what we wanted<br />

we did not know what we were doing<br />

there was no accounting experience within the firm<br />

we could not implement <strong>on</strong> our own<br />

insufficient IS expertise<br />

need advice <strong>on</strong> software soluti<strong>on</strong> and assistance to install the software<br />

5. The tasks and duties of c<strong>on</strong>sultants<br />

The c<strong>on</strong>sultants engaged in a variety of tasks for each project. Table 4 provides a summary of the<br />

major tasks that the c<strong>on</strong>sultants performed at each of the four firms. Table 4 also shows that the<br />

c<strong>on</strong>sultants were involved at different stages in the overall life cycle of the project, based <strong>on</strong> the<br />

systems development life cycle (SDLC). The most comm<strong>on</strong> tasks carried out by c<strong>on</strong>sultants were:<br />

analysing needs and recommending a system, installing software, and c<strong>on</strong>figuring software.<br />

Table 4: Tasks carried out by c<strong>on</strong>sultants at the SMEs<br />

SMEs Duties and tasks performed by c<strong>on</strong>sultant SDLC Phase<br />

SP<br />

AB<br />

AR<br />

DL<br />

6. IS competences<br />

Analysis & Recommendati<strong>on</strong>s<br />

Installati<strong>on</strong> of the software<br />

C<strong>on</strong>figurati<strong>on</strong> of the software<br />

Customisati<strong>on</strong> of the software<br />

Training users<br />

On-going technical support<br />

Analysis & Recommendati<strong>on</strong>s<br />

Installati<strong>on</strong> of the software<br />

C<strong>on</strong>figurati<strong>on</strong> of the software<br />

On-going accounting functi<strong>on</strong><br />

Installati<strong>on</strong> of the software<br />

C<strong>on</strong>figurati<strong>on</strong> of the software<br />

Customisati<strong>on</strong> of the software<br />

Analysis & Recommendati<strong>on</strong>s<br />

Installati<strong>on</strong> of the software<br />

Training users<br />

Analysis & Design<br />

Implementati<strong>on</strong><br />

Implementati<strong>on</strong><br />

Implementati<strong>on</strong><br />

Implementati<strong>on</strong><br />

Support<br />

Analysis & Design<br />

Implementati<strong>on</strong><br />

Implementati<strong>on</strong><br />

Support<br />

Implementati<strong>on</strong><br />

Implementati<strong>on</strong><br />

Implementati<strong>on</strong><br />

Implementati<strong>on</strong><br />

Implementati<strong>on</strong><br />

Implementati<strong>on</strong><br />

The interviews provided evidence that SMEs lack IS skills and abilities. Some examples of this are<br />

provided in Table 5, based <strong>on</strong> the competences framework of Cragg et al (2011). The SME managers<br />

indicated a preference to focus <strong>on</strong> their business and leave many IS activities to the experts. The<br />

SMEs also lack the ability to solve problems. The c<strong>on</strong>sultants noted that some SMEs would be aware<br />

they have an IS-related problem but would not understand the problem. SMEs also lack knowledge of<br />

accounting soluti<strong>on</strong>s to the extent that they do not know what system would meet their needs. SMEs<br />

usually followed the recommendati<strong>on</strong>s of c<strong>on</strong>sultants <strong>on</strong> what soluti<strong>on</strong> they would implement.<br />

Implementing accounting software involves more than just installing the software as the system must<br />

be aligned with processes within the business. The interviews provided evidence that SMEs were<br />

unable to do this. In some cases the accounting package was modified to fit the business. In other<br />

cases, the new accounting package changed the way the business operated and the c<strong>on</strong>sultant<br />

played an important role in assisting the organisati<strong>on</strong> with the changes to their processes. The SMEs<br />

managed their implementati<strong>on</strong> project internally. However, as <strong>on</strong>e c<strong>on</strong>sultant pointed out, c<strong>on</strong>sultants<br />

occasi<strong>on</strong>ally assist SMEs with managing implementati<strong>on</strong> projects. All of the SMEs had limited<br />

knowledge of the infrastructure requirements of their accounting systems. The c<strong>on</strong>sultants were the<br />

<strong>on</strong>es that exhibited these abilities and advised the organisati<strong>on</strong>s <strong>on</strong> infrastructure, particularly the<br />

hardware and networks needed to operate systems effectively.<br />

28


Adrian Bradshaw, Paul Cragg and Venkat Pulakanam<br />

Table 5: The respective roles played by SMEs and c<strong>on</strong>sultants<br />

Macrocompetence<br />

Competence Role of SMEs Role of C<strong>on</strong>sultants<br />

Business Process<br />

<strong>Management</strong><br />

C<strong>on</strong>sultants assist SMEs Assist SMEs through advice<br />

Define IS<br />

Define IS C<strong>on</strong>sultants assist SMEs Assist SMEs to align IS with<br />

C<strong>on</strong>tributi<strong>on</strong> Requirements<br />

business<br />

Accessing IS Most SMEs seek advice Provide informati<strong>on</strong> and advice<br />

Knowledge from outside sources<br />

<strong>on</strong> IS<br />

Define IS<br />

Strategy<br />

Technology<br />

Infrastructure<br />

Requirements<br />

There was existing<br />

infrastructure in some<br />

cases<br />

Provide informati<strong>on</strong>, advice and<br />

recommendati<strong>on</strong>s <strong>on</strong><br />

infrastructure<br />

Benefits <strong>Management</strong> C<strong>on</strong>sultants assist SMEs Aid SMEs to assess IS benefits<br />

Managing Change C<strong>on</strong>sultants assist SMEs Assist with change<br />

Exploitati<strong>on</strong><br />

management<br />

Project <strong>Management</strong> In some cases SMEs Manage the projects in some<br />

manage the projects.<br />

cases<br />

Implementati<strong>on</strong> and Carried out by the Carried out by the c<strong>on</strong>sultants<br />

Deliver<br />

integrati<strong>on</strong><br />

c<strong>on</strong>sultants<br />

Soluti<strong>on</strong>s Business C<strong>on</strong>tinuity &<br />

Security<br />

C<strong>on</strong>sultants assist SMEs Carried out by the c<strong>on</strong>sultants<br />

Manage IS Supplier C<strong>on</strong>sultants worked with C<strong>on</strong>sultants actively seek to<br />

Relati<strong>on</strong>ships<br />

SMEs to develop build relati<strong>on</strong>ships with SMEs<br />

Supply<br />

Staff development<br />

relati<strong>on</strong>ships.<br />

Staff were not formally Provide training <strong>on</strong> IS and other<br />

trained, they learned <strong>on</strong><br />

the job.<br />

related business areas<br />

7. Phase 2<br />

To examine the competences perspective further, a sec<strong>on</strong>d round of interviews were c<strong>on</strong>ducted with<br />

both SMEs and c<strong>on</strong>sultants. These interviews focused <strong>on</strong> the third research questi<strong>on</strong>, ie:<br />

Determine if c<strong>on</strong>sultants influence IS competences in SMEs.<br />

The aim of these interviews was to focus <strong>on</strong> how IS competences may have been influenced by<br />

c<strong>on</strong>sultants during the implementati<strong>on</strong> project. The interview questi<strong>on</strong>s and the analysis drew <strong>on</strong> the<br />

framework of IS competences (Cragg et al, 2011). Each SME case was examined to identify<br />

instances where c<strong>on</strong>sultants influenced (ie, had an impact <strong>on</strong> the creati<strong>on</strong> or use of) any of the listed<br />

competences. The cases provided str<strong>on</strong>g evidence that c<strong>on</strong>sultants help SMEs overcome many IS<br />

competences that SMEs lack. Another c<strong>on</strong>clusi<strong>on</strong> is that c<strong>on</strong>sultants influenced all six of the macro<br />

competences of Cragg et al (2011), ie:<br />

Business and IS strategic thinking<br />

Define IS c<strong>on</strong>tributi<strong>on</strong><br />

Define the IS strategy<br />

Exploitati<strong>on</strong><br />

Deliver soluti<strong>on</strong>s<br />

Supply<br />

The Business and IS strategic thinking competence is defined as “an organizati<strong>on</strong>’s ability to identify<br />

and evaluate the need for IS in providing opportunities to develop a better business strategy and to<br />

manage the IS activities effectively, including establishing an appropriate IS organizati<strong>on</strong> and defining<br />

roles, resp<strong>on</strong>sibilities and policies” (Cragg et al., 2011, p.356). This relates to knowledge about how<br />

accounting software can be of value to a business (Eikebrokk and Olsen, 2007). It relates to the ability<br />

to define a business case and establish appropriate criteria for making decisi<strong>on</strong>s about IS (Cragg et al<br />

2011). The interviews revealed that c<strong>on</strong>sultants help SMEs understand the value of IS, including the<br />

implicati<strong>on</strong>s of implementing IS and in some instances helping SMEs to establish the business case<br />

for the project. This is because some SMEs lack the ability to establish a formal business case for<br />

implementing IS. C<strong>on</strong>sultants provide a means for SMEs to identify and evaluate the potential and<br />

implicati<strong>on</strong>s of implementing an accounting package. The c<strong>on</strong>sultants share knowledge with SMEs <strong>on</strong><br />

the potential of implementing particular software. The ability to identify and evaluate the potential and<br />

implicati<strong>on</strong>s of implementing accounting software (Cragg et al., 2011) relates to knowledge of how<br />

29


Adrian Bradshaw, Paul Cragg and Venkat Pulakanam<br />

accounting software can be of value to the organisati<strong>on</strong> (Eikibrokk and Olsen, 2007). In providing this<br />

ability to SMEs, c<strong>on</strong>sultants compensate for/overcome the SME’s lack of ability. The interviews<br />

revealed support for the influence of c<strong>on</strong>sultants <strong>on</strong> SMEs understanding the value of implementing<br />

AIS. The interviews indicated that c<strong>on</strong>sultants enhanced the ability of SMEs to define the potential<br />

and implicati<strong>on</strong>s of implementing AIS.<br />

Define IS C<strong>on</strong>tributi<strong>on</strong> refers to the ability to “translating the business strategy into investments in IS<br />

that achieve both performance improvements and meet informati<strong>on</strong> needs” (Cragg et al., 2011,<br />

p.358). It includes four competences relating to alignment, business process management, defining IS<br />

requirements, and accessing IS knowledge. The discussi<strong>on</strong>s revealed that c<strong>on</strong>sultants assist SMEs in<br />

managing business processes related to the implementati<strong>on</strong> of accounting packages. One c<strong>on</strong>sultant<br />

commented: “I've come across a lot of clients where they have never d<strong>on</strong>e the books in-house; the<br />

accountant handles everything. The accountant does all their processing; they just provide bank<br />

statements or they provide informati<strong>on</strong> to the accountant; he does everything. They get to a point of<br />

saying, “Weren’t you want to do this ourselves? So therefore, then; their system, at the moment; they<br />

d<strong>on</strong>'t have a system, basically.” C<strong>on</strong>sultants also influence SMEs’ ability to access IS knowledge,<br />

primarily when it comes to finding hardware providers.<br />

The Define the IS Strategy macro-competence is the ability to “define the informati<strong>on</strong> and applicati<strong>on</strong><br />

architectures, technology infrastructure and IS resources it needs to enable the resources to be<br />

successfully bought and/or implemented (Cragg et al., 2011, p.358). It addresses three IS abilities: to<br />

define an appropriate software sourcing strategy, appropriate IS acquisiti<strong>on</strong> process, and appropriate<br />

technology infrastructure. This relates to two competences from Eikebrokk and Olsen (2007), ie,<br />

sourcing and systems and infrastructure. If SMEs have these abilities, then it is likely that the<br />

implemented IS will be useable and have a positive impact <strong>on</strong> IS success. Package acquisiti<strong>on</strong> is the<br />

comm<strong>on</strong> sourcing strategy for software, while the hardware infrastructure competence requires an<br />

understanding of the infrastructure needed to implement the software. It includes computing<br />

hardware, software and network infrastructure needed to implement a working system. The interviews<br />

indicated that SMEs follow the recommendati<strong>on</strong>s of external parties like c<strong>on</strong>sultants and<br />

accountants. SMEs often lack the ability to evaluate the various software soluti<strong>on</strong>s and do not<br />

understand the infrastructure requirements of IS they wish to implement. C<strong>on</strong>sultants play a key role<br />

by assisting SMEs with sourcing and acquiring IS. The c<strong>on</strong>sultant also assists clients to evaluate<br />

opti<strong>on</strong>s and recommends which system is best suited for their needs. They make an assessment of<br />

what the client requires and advise them <strong>on</strong> the software to use, typically an off-the-shelf system. For<br />

example, if MYOB is not suitable, the c<strong>on</strong>sultant will suggest an alternative. In additi<strong>on</strong> to software,<br />

c<strong>on</strong>sultants typically either recommend new hardware or the upgrading of the existing infrastructure.<br />

One c<strong>on</strong>sultant explained that she outlines what hardware infrastructure is needed to implement a<br />

system. However, the logistics surrounding the hardware and infrastructure are left to SMEs and their<br />

IT suppliers. The c<strong>on</strong>sultant liaises with the IT suppliers. Sometimes a c<strong>on</strong>sultant may recommend an<br />

SME seek guidance from a hardware specialist, ie, another c<strong>on</strong>sultant from their "trusted circle".<br />

The Exploitati<strong>on</strong> competence relates “to the organizati<strong>on</strong>’s ability to increase the benefits from<br />

effective use of informati<strong>on</strong> and applicati<strong>on</strong> investments” (Cragg et al., p.359). It includes four<br />

competences: benefits management, managing change, project management and inter-organizati<strong>on</strong>al<br />

collaborati<strong>on</strong>. The discussi<strong>on</strong>s targeted the ability of SMEs to exploit accounting systems. Training<br />

was identified as aiding clients to maximise the benefits of the system by learning tricks of how to use<br />

the software. C<strong>on</strong>sultants also assist clients in maximising their system by bringing organisati<strong>on</strong>s<br />

closer to their accountants. The c<strong>on</strong>sultant recognised that accountants can improve business<br />

development of an SME. The c<strong>on</strong>sultants also recognised that changes must be made to business<br />

processes in order to derive additi<strong>on</strong>al benefits. However, such changes cannot be introduced<br />

immediately after implementati<strong>on</strong>, since the clients must be allowed to work in their usual manner and<br />

be comfortable. Only after that was achieved, would the c<strong>on</strong>sultant suggest changes, “what we tend<br />

to do is put the system in, let them get used to doing the things the way they’ve always d<strong>on</strong>e them<br />

and then they’ll say, we want to bring in job management. We’ve always partially d<strong>on</strong>e it but not<br />

quite.”<br />

The Deliver Soluti<strong>on</strong>s competence c<strong>on</strong>cerns the ability “to c<strong>on</strong>vert requirements into working IS<br />

assets (business soluti<strong>on</strong>s) that perform according to specificati<strong>on</strong> and can be integrated effectively<br />

with other systems and processes” (Cragg et al., 2011, p.359). In includes four competences:<br />

applicati<strong>on</strong>s development, implementati<strong>on</strong> and integrati<strong>on</strong>, apply and use technology, and business<br />

30


Adrian Bradshaw, Paul Cragg and Venkat Pulakanam<br />

c<strong>on</strong>tinuity and security. The SMEs in this study were not in a positi<strong>on</strong> to carry out IS implementati<strong>on</strong><br />

nor integrati<strong>on</strong> without the assistance of c<strong>on</strong>sultants. Instead, c<strong>on</strong>sultants undertook such tasks as<br />

part of the IS implementati<strong>on</strong> project. They typically installed the software without having to do any<br />

major customisati<strong>on</strong>. Any customisati<strong>on</strong> would typically involve reports and invoices. The c<strong>on</strong>sultants<br />

indicated that the encouraged SMEs to apply and follow regular back-up procedures; unfortunately<br />

this advice was seldom followed. One c<strong>on</strong>sultant indicated that she teaches SMEs the process of not<br />

<strong>on</strong>ly backing up the accounting system, but also other areas like email.<br />

The Supply competence refers to three “operati<strong>on</strong>al competences that allow the organizati<strong>on</strong> to<br />

create and maintain its technology resources and applicati<strong>on</strong>s through effective management of the<br />

IS supply chain and internal and external IS resources” (Cragg et al., 2011, p.360). It includes<br />

managing IS suppliers, asset management, and staff development. A theme to emerge from the<br />

interviews is that c<strong>on</strong>sultants develop <strong>on</strong>-going relati<strong>on</strong>ships with their clients. The c<strong>on</strong>sultants<br />

c<strong>on</strong>tend that they had excellent relati<strong>on</strong>ships with most of their clients, with <strong>on</strong>e likening it being<br />

similar to a friendship. Another claimed to “keep an eye <strong>on</strong> things.” These <strong>on</strong>-going relati<strong>on</strong>s allow<br />

c<strong>on</strong>sultants to assist SMEs in keeping their systems operati<strong>on</strong>al. C<strong>on</strong>sultants also teach or explain to<br />

clients the importance of maintaining their systems, including keeping software up to date.<br />

C<strong>on</strong>sultants also worked to ensure that the technical skills within SMEs were adequate for the needs<br />

of the organisati<strong>on</strong>. By so doing c<strong>on</strong>sultants build/enhance the ability of SMEs to utilise IS. After<br />

implementati<strong>on</strong>, SMEs learn what requirements are more suited to their type of organisati<strong>on</strong> and<br />

operati<strong>on</strong>. They learn how to use or integrate the use of IS with their current processes. Over time,<br />

SMEs become more capable in areas related to the integrati<strong>on</strong> of IS with the organisati<strong>on</strong>’s business<br />

processes.<br />

8. Discussi<strong>on</strong> and c<strong>on</strong>clusi<strong>on</strong>s<br />

The SMEs in this study did not have appropriate IS skills and abilities to select and implement a major<br />

new system. Thus the SMEs engaged c<strong>on</strong>sultants due to a lack of IS competences. It seems that the<br />

c<strong>on</strong>sultants compensated for a lack of competences within the SMEs, in line with Nevo et al (2007).<br />

The study c<strong>on</strong>tributes by identifying numerous competences that were compensated for by<br />

c<strong>on</strong>sultants. The study thus adds to our understanding of c<strong>on</strong>sultants acting as intermediaries to<br />

assist and advise firms, as argued by Bessant and Rush (1995). In this intermediary role as ‘c<strong>on</strong>duits’,<br />

c<strong>on</strong>sultants provide advice to assist SMEs with finding appropriate products, implementing the<br />

system, integrating software with existing systems, and training and support (Howcroft and Light,<br />

2008). The reas<strong>on</strong>s for SMEs engaging c<strong>on</strong>sultants reflect the resource-poor nature of many SMEs,<br />

as discussed by much prior literature <strong>on</strong> small firms.<br />

Prior SME-based research has identified numerous IS competences (Scupola, 2008; Eikebrokk and<br />

Olsen, 2007; Cragg et al., 2011). This study c<strong>on</strong>tributes to this research by identifying IS<br />

competences that are relevant during the implementati<strong>on</strong> of IS and, importantly, identifies many<br />

competences that are lacking in SMEs. This study provides evidence that c<strong>on</strong>sultants affected all six<br />

of the IS macro competences proposed by Cragg et al (2011). Disappointingly, for this small sample<br />

of SMEs, few if any internal competences were improved during the c<strong>on</strong>sultant engagement process.<br />

It seems possible that many SMEs are failing to take the opportunity to learn from c<strong>on</strong>sultants.<br />

Instead they rely heavily <strong>on</strong> the c<strong>on</strong>sultants, and typically follow their advice. SMEs could recognise<br />

that c<strong>on</strong>sultants are a source of IS competences. Some SMEs could aim to learn from c<strong>on</strong>sultants to<br />

help build, enhance or improve internal IS competences.<br />

Prior research has identified dangers in SMEs relying heavily <strong>on</strong> c<strong>on</strong>sultants. For example, Howcroft<br />

and Light (2008) expressed the c<strong>on</strong>cern that the c<strong>on</strong>sultants may gain more than the SME. Thus, if<br />

SMEs decide to rely heavily <strong>on</strong> c<strong>on</strong>sultants, they need to develop their ability to manage this<br />

relati<strong>on</strong>ship. Cragg et al (2011) referred to this as the ability to ‘manage IS supplier relati<strong>on</strong>ships’, ie,<br />

to “develop value added relati<strong>on</strong>ships between the business and IS suppliers (external and internal),<br />

including service level agreements and c<strong>on</strong>tract management (performance m<strong>on</strong>itoring, problem<br />

resoluti<strong>on</strong> and negotiating amendments) (p.357). It seems likely that some SMEs will c<strong>on</strong>tinue to rely<br />

heavily <strong>on</strong> c<strong>on</strong>sultants. An implicati<strong>on</strong> for SMEs is that they will need to find ways to develop this<br />

competence. Further research could examine how SMEs achieve an ability to manage supplier<br />

relati<strong>on</strong>ships, and whether this competence influences IS success in SMEs.<br />

It should be noted that this study was limited to a relatively small number of SMEs seeking a new<br />

accounting package. Thus a different set of SMEs and c<strong>on</strong>sultants, and a different applicati<strong>on</strong> system<br />

31


Adrian Bradshaw, Paul Cragg and Venkat Pulakanam<br />

could present different results. Also, the list of competences may not be an exhaustive list as different<br />

frameworks indicate different competences. This study focused <strong>on</strong> competences that are applicable to<br />

IS implementati<strong>on</strong> projects.<br />

References<br />

Basil, P., Yen, D., and Tang, H. (1997). Informati<strong>on</strong> c<strong>on</strong>sulting: developments, trends and suggesti<strong>on</strong>s for growth.<br />

Internati<strong>on</strong>al Journal of Informati<strong>on</strong> <strong>Management</strong>, 17(5), 303-323.<br />

Bessant, J., and Rush, H. (1995). Building bridges for innovati<strong>on</strong>: the role of c<strong>on</strong>sultants in technology transfer.<br />

Research Policy, 24(1), 97-114.<br />

Bruque, S. and Moyano, J. (2007). Organisati<strong>on</strong>al determinants of informati<strong>on</strong> technology adopti<strong>on</strong> and<br />

implementati<strong>on</strong> in SMEs: The case of family and cooperative firms. Technovati<strong>on</strong>, 27:5, 241-253.<br />

Butler, T., and Murphy, C. (2008). An exploratory study <strong>on</strong> IS capabilities and assets in a small-to-medium<br />

software enterprise. Journal of Informati<strong>on</strong> Technology, 23(4), 330-344.<br />

Caldeira, M.M. and Ward, J.M. (2003). Using resource-based theory to interpret the successful adopti<strong>on</strong> and use<br />

of informati<strong>on</strong> systems and technology in manufacturing small and medium-sized enterprises, <str<strong>on</strong>g>European</str<strong>on</strong>g><br />

Journal of Informati<strong>on</strong> Systems, 12(2), 127-141.<br />

Carey, J. (2008). Role misc<strong>on</strong>cepti<strong>on</strong>s and negotiati<strong>on</strong>s in small business owner/web developer relati<strong>on</strong>ships.<br />

Journal of <strong>Management</strong> & Organizati<strong>on</strong>, 14(1), 85-99.<br />

Champi<strong>on</strong>, D. P., Kiel, D. H., and McLend<strong>on</strong>, J. A. (1990). Choosing a c<strong>on</strong>sulting role. Training and Development<br />

Journal, 44(2), 66.<br />

Cragg, P., Caldeira, M. and Ward, J. (2011). Organizati<strong>on</strong>al Informati<strong>on</strong> Systems Competences in Small and<br />

Medium-sized Enterprises. Informati<strong>on</strong> & <strong>Management</strong>, 48:8, 353-363.<br />

de Guinea, A. O., Kelley, H., and Hunter, M. G. (2005). Informati<strong>on</strong> Systems Effectiveness in Small Businesses:<br />

Extending a Singaporean Model in Canada. Journal of Global Informati<strong>on</strong> <strong>Management</strong>, 13(3), 55-79.<br />

Eikebrokk, T.R. and Olsen, D.H. (2007) An empirical investigati<strong>on</strong> of competency factors affecting e-business<br />

success in <str<strong>on</strong>g>European</str<strong>on</strong>g> SMEs. Informati<strong>on</strong> & <strong>Management</strong>, 44(4), 364-383.<br />

Eisenhardt, K. (1989). Building theories from case study research. Academy <strong>Management</strong> Review, 14:4, 532-<br />

550.<br />

Howcroft, D., and Light, B. (2008). IT c<strong>on</strong>sultants, salesmanship and the challenges of packaged software<br />

selecti<strong>on</strong> in SMEs. Journal of Enterprise Informati<strong>on</strong> <strong>Management</strong>, 21(6), 597-615.<br />

Nevo, S., Wade, M., and Cook, W. (2007). An examinati<strong>on</strong> of the trade-off between internal and external IT<br />

capabilities. Journal of Strategic Informati<strong>on</strong> Systems, 16:1, 5-23.<br />

Scupola, A. (2008). C<strong>on</strong>ceptualizing Competences in e-services adopti<strong>on</strong> and assimilati<strong>on</strong> in SMEs. Journal of<br />

Electr<strong>on</strong>ic Commerce in Organizati<strong>on</strong>s, 6(2), 78-91.<br />

Soh, C. P. P., Yap, C. S., and Raman, K. S. (1992). Impact of c<strong>on</strong>sultants <strong>on</strong> computerizati<strong>on</strong> success in small<br />

businesses. Informati<strong>on</strong> and <strong>Management</strong>, 22(5), 309-319.<br />

Th<strong>on</strong>g, J. Y. L. (2001). Resource c<strong>on</strong>straints and informati<strong>on</strong> systems implementati<strong>on</strong> in Singaporean small<br />

businesses. Omega, 29(2), 143-156.<br />

Th<strong>on</strong>g, J. Y. L., Yap, C. S., and Raman, K. S. (1994). Engagement of external expertise in informati<strong>on</strong> systems<br />

implementati<strong>on</strong>. Journal of <strong>Management</strong> Informati<strong>on</strong> Systems, 11(2), 209-231.<br />

Th<strong>on</strong>g, J. Y. L., Yap, C. S., and Raman, K. S. (1997). Envir<strong>on</strong>ments for Informati<strong>on</strong> Systems Implementati<strong>on</strong> in<br />

Small Businesses. Journal of Organizati<strong>on</strong>al Computing and Electr<strong>on</strong>ic Commerce, 7(4), 253-278.<br />

Yin, R. (2009). Case study research: design and methods. Sage.<br />

32


Developing a Framework for Maturing IT Risk <strong>Management</strong><br />

Capabilities<br />

Marian Carcary<br />

Innovati<strong>on</strong> Value Institute, Nati<strong>on</strong>al University of Ireland Maynooth, Maynooth,<br />

Co Kildare, Ireland<br />

marian.carcary@nuim.ie<br />

Abstract: Understanding the value derived from IT investments and IT enabled operati<strong>on</strong>al improvements is<br />

difficult, and has been a subject of research and debate am<strong>on</strong>g ICT practiti<strong>on</strong>ers and academics for many years.<br />

This is particularly so because innovative technological developments have supported transformative changes in<br />

organizati<strong>on</strong>al operati<strong>on</strong>al activities. Research c<strong>on</strong>tinues to investigate approaches to not <strong>on</strong>ly understanding the<br />

value derived by IT but also to optimizing this value. One of the key aspects of optimizing IT-driven value is the<br />

requirement to effectively manage risk. The c<strong>on</strong>tinual evoluti<strong>on</strong> of the IT risk landscape requires effective Risk<br />

<strong>Management</strong> (RM) practices for all IT risk areas, such as, but not limited to security, investments, service<br />

c<strong>on</strong>tracts, data protecti<strong>on</strong> and informati<strong>on</strong> privacy. Effectively managing these risk areas pose specific c<strong>on</strong>cerns<br />

from the perspective of Chief Informati<strong>on</strong> Officers (CIOs) and Chief Risk Officers (CROs). Hence, significant<br />

c<strong>on</strong>siderati<strong>on</strong>s should be given to not <strong>on</strong>ly the processes involved in assessing, prioritizing, handling and<br />

m<strong>on</strong>itoring these risks but also to ensuring the development of an appropriate risk culture and the establishment<br />

of effective RM governance structures, to support effective RM. This paper examines the maturity<br />

model/framework approach to improving an organizati<strong>on</strong>’s IT capabilities, with specific reference to effectively<br />

managing IT-related risks, and increasing value derived over time. A new IT Risk <strong>Management</strong> maturity model is<br />

presented; this framework is part of the IT Capability Maturity Framework (IT CMF) which supports value-driven<br />

IT management practices. It was developed by the Innovati<strong>on</strong> Value Institute at the Nati<strong>on</strong>al University of Ireland<br />

Maynooth, following a design science and open innovati<strong>on</strong> research approach. The IT CMF, c<strong>on</strong>sisting of 33<br />

Critical Capabilities, focuses <strong>on</strong> maturing key activities of the IT organizati<strong>on</strong>. The Risk <strong>Management</strong> Critical<br />

Capability presented in this paper enables organizati<strong>on</strong>s to determine their IT RM maturity and identify key<br />

recommendati<strong>on</strong>s in specific areas to improve maturity overtime. Thereafter the paper presents an analysis of the<br />

maturity model approach to managing risk, to improving an organizati<strong>on</strong>’s IT capabilities, and to deriving<br />

enterprise-wide value from more mature IT practices.<br />

Keywords: IT risks, IT risk management, maturity model, IT CMF, critical capability<br />

1. Introducti<strong>on</strong><br />

Risk is a functi<strong>on</strong> of the likelihood of a particular threat source exploiting an organizati<strong>on</strong>’s<br />

vulnerability, and the impact of the adverse event <strong>on</strong> the organizati<strong>on</strong> (Elky, 2006). However, for many<br />

organizati<strong>on</strong>s the various IT risks are often under assessed (Benaroch et al, 2006; Glass, 2006). As<br />

technology c<strong>on</strong>tinues to drive industry transformati<strong>on</strong>, traditi<strong>on</strong>al business models are gradually being<br />

replaced by technology-enabled models (Ernst and Young, 2011), and while this may support<br />

improved operati<strong>on</strong>al efficiency, it also exposes an organizati<strong>on</strong> to increased risk likelihood and<br />

impact levels. Today, with the proliferati<strong>on</strong> of mobile computing, social networking, and cloud based<br />

services, organizati<strong>on</strong>s face increased risk of data leakage, asset theft and reputati<strong>on</strong>al damage. In<br />

fact, IT risks stories are comm<strong>on</strong> in the recent literature. Reports of the TK Maxx security breach<br />

resulting in theft of over 45 milli<strong>on</strong> customer card numbers (Gaudin, 2007); Est<strong>on</strong>ia’s denial of service<br />

attacks, affecting government, banking and school websites (Kirk, 2007); and the recent high-profile<br />

wiki-leaks publishing global intelligence files are just a few examples.<br />

Therefore, the ability to effectively manage the various IT risks is an important factor in organizati<strong>on</strong>s<br />

deriving and optimizing the value associated with their IT investments. Effective practices should<br />

c<strong>on</strong>sider all key IT risk areas to enable CIOs and CROs to prioritize their resources in addressing the<br />

most significant risks. This paper presents a new maturity modeling approach to identifying and<br />

developing an organizati<strong>on</strong>’s risk management capabilities. The maturity model in Informati<strong>on</strong><br />

Systems (IS) research c<strong>on</strong>tinues to grow in popularity, and while c<strong>on</strong>cerns exist regarding the<br />

development process and foundati<strong>on</strong>s up<strong>on</strong> which some models are developed, the RM capability<br />

maturity model presented in this paper was built up<strong>on</strong> existing theories and methodologies, followed a<br />

rigorous development process based <strong>on</strong> a design science approach, and was externally validated in a<br />

number of pilot organizati<strong>on</strong>s.<br />

The structure of the paper is as follows: Secti<strong>on</strong> 2 presents an overview of the IT risk landscape and<br />

existing approaches to RM. Secti<strong>on</strong> 3 introduces the c<strong>on</strong>cepts of maturity models in IS research and<br />

highlights the c<strong>on</strong>cerns that exist regarding the approach. Secti<strong>on</strong> 4 provides an overview of a new IT<br />

33


Marian Carcary<br />

management maturity model, the IT Capability Maturity Framework (IT CMF), and an outline of how<br />

the c<strong>on</strong>cerns associated with maturity models were addressed in its development. It further discusses<br />

the model’s RM critical capability for assessing and improving RM maturity overtime. Secti<strong>on</strong> 5<br />

c<strong>on</strong>cludes the paper with a discussi<strong>on</strong> of the value of the maturity modeling approach for optimizing IT<br />

capabilities, and specifically RM capabilities<br />

2. Managing IT risks<br />

Investing in IT exposes an organisati<strong>on</strong> to several risk factors, including for example project,<br />

organizati<strong>on</strong>al and technical risks (see for example Amberg and Okujava, 2005; Brown, 2005).<br />

Undoubtedly, <strong>on</strong>e of the biggest c<strong>on</strong>cerns from an organizati<strong>on</strong>s perspective is security, in terms of<br />

protecting the organizati<strong>on</strong>s business critical applicati<strong>on</strong>s and c<strong>on</strong>fidential/sensitive data. The Frost<br />

and Sullivan (2011) study, which was c<strong>on</strong>ducted for the Internati<strong>on</strong>al Informati<strong>on</strong> Systems Security<br />

Certificati<strong>on</strong>s C<strong>on</strong>sortium (ISC 2 ), reported that key risks from an organizati<strong>on</strong>s security perspective<br />

include applicati<strong>on</strong> vulnerabilities, mobile devices, viruses and worm attacks, internal employees,<br />

hackers, c<strong>on</strong>tractors, cyber terrorism, cloud-based services and organized crime. The study further<br />

reported that the key new and emerging risks facing organizati<strong>on</strong>s today include mobile devices and<br />

mobility, cloud computing and social media.<br />

Advancements from PDAs to multi-functi<strong>on</strong>al and ubiquitous smartph<strong>on</strong>es and tablets have resulted<br />

in a proliferati<strong>on</strong> of mobile devices. However, ability to access business applicati<strong>on</strong>s, corporate<br />

sensitive data and c<strong>on</strong>fidential pers<strong>on</strong>al data “anytime, anywhere” poses risks regarding data leaks or<br />

loss/theft of mobile devices. For example, smartph<strong>on</strong>es were growing at the rate of 21% in North<br />

America, and tablets and e-readers were expected to reach sales levels of 22 milli<strong>on</strong> units in North<br />

America by 2016. This c<strong>on</strong>cept of the “borderless envir<strong>on</strong>ment” poses specific c<strong>on</strong>cerns from a data<br />

security and c<strong>on</strong>trol perspective (Frost and Sullivan, 2011; Ernst and Young, 2011).<br />

Cloud computing, regarded as an enabler of scalable, flexible and powerful computing, poses specific<br />

c<strong>on</strong>cerns in terms of c<strong>on</strong>fidential informati<strong>on</strong> exposure to unauthorized sources; loss or leakage of<br />

c<strong>on</strong>fidential data; weak systems or applicati<strong>on</strong> c<strong>on</strong>trols; susceptibility to cyber-attacks; disrupti<strong>on</strong>s in<br />

the c<strong>on</strong>tinuous operati<strong>on</strong>s of the data centre; and inability to support compliance audits, am<strong>on</strong>g others<br />

(Frost and Sullivan, 2011). Similar cloud based challenges and a number of additi<strong>on</strong>al <strong>on</strong>es were<br />

highlighted in Ernst and Young’s (2011) Global Informati<strong>on</strong> Security survey and include legal<br />

compliance and privacy; informati<strong>on</strong> security and data integrity; c<strong>on</strong>tractual and legal risks;<br />

governance and risk management assurance; reliability and c<strong>on</strong>tinuity of operati<strong>on</strong>s; and integrati<strong>on</strong><br />

and interoperability. From the Informati<strong>on</strong> System’s Audit and C<strong>on</strong>trol Associati<strong>on</strong>’s (ISACA) (2010)<br />

survey, 45% of US IT professi<strong>on</strong>als believed that the risks of cloud adopti<strong>on</strong> outweighed any<br />

associated benefits; <strong>on</strong>ly 10% surveyed would c<strong>on</strong>sider migrating missi<strong>on</strong> critical applicati<strong>on</strong>s to the<br />

cloud. However, 61% of Ernst and Young’s (2011) resp<strong>on</strong>dents were currently using, evaluating or<br />

planning adopti<strong>on</strong> of cloud computing-based services.<br />

Further, the growth in use of social media tools means that social media applicati<strong>on</strong>s are now being<br />

used, not just for pers<strong>on</strong>al uses but also business purposes, in c<strong>on</strong>necting with customers, tracking<br />

customer comments about their products and services, developing brand loyalty etc. Approximately<br />

15% of the world’s populati<strong>on</strong> are registered users of popular social and business networking sites.<br />

For example, Facebook had 687.1 milli<strong>on</strong> users in June, 2011, while LinkedIn had 79.2 milli<strong>on</strong> unique<br />

visitors worldwide in March 2011. IT risks associated with their use for business purposes include<br />

exposure to malicious software within social networks; hacked accounts; and exposure of c<strong>on</strong>fidential<br />

data or sensitive company informati<strong>on</strong> (Ernst and Young, 2011).<br />

However, IT risks span a broader spectrum than security, and include a wide range of risks that may<br />

affect or result from IT operati<strong>on</strong>s, for example risks associated with compliance with regulatory<br />

changes; compliance with ethics policies; IT investments; IT project lifecycles; service c<strong>on</strong>tinuity due<br />

to security breaches, system failure or natural disasters; internal process changes impacting <strong>on</strong><br />

product or service quality; supplier c<strong>on</strong>tracts; and reputati<strong>on</strong>. Hence, an effective approach to<br />

managing these and other IT risks is required to enable organizati<strong>on</strong>s to reduce their exposure and<br />

their potential impact <strong>on</strong> the organizati<strong>on</strong>’s operati<strong>on</strong>s and in essence to protect the organizati<strong>on</strong>’s<br />

assets and missi<strong>on</strong> (Elky, 2006). Some of the various IT risks may be intractable, in that they resist<br />

mitigating acti<strong>on</strong>s, or unforeseen/ not apparent at the time of project planning (Taylor, 2006). Hence, a<br />

proactive approach to identifying and scoring IT risks including new and emerging risks, to prioritizing<br />

identified risks according to determined risk likelihood and impact scores, to identifying and<br />

34


Marian Carcary<br />

implementing appropriate risk handling strategies, and to m<strong>on</strong>itoring effectiveness of the implemented<br />

risk c<strong>on</strong>trols overtime is required.<br />

<strong>Management</strong> of risk is well discussed in the literature (for example Casey, 2007; Rosenquist, 2007;<br />

Westerman and Hunter, 2007), and several IT management frameworks address the issue of RM in<br />

varying degrees of depth (for example, CMMI, <strong>Management</strong> of Risk (MoR), ISO 27001, ISO 27002, IT<br />

Risk Framework, and COBIT). “Risk management is the process that allows IT managers to balance<br />

the operati<strong>on</strong>al and ec<strong>on</strong>omic costs of protective measures and achieve gains in missi<strong>on</strong> capability by<br />

protecting the IT systems and data that support their organizati<strong>on</strong>s’ missi<strong>on</strong>s” (St<strong>on</strong>eburner et al,<br />

2002). Ability to understand IT risks <strong>on</strong> the horiz<strong>on</strong> and the likelihood and magnitude of these risks<br />

enables stakeholders to prioritize scare resources and take steps to protect the IT assets<br />

proporti<strong>on</strong>ate to their value to the organizati<strong>on</strong>. Strategies for managing risk, such as mitigati<strong>on</strong>,<br />

transference, acceptance, or avoidance, will depend <strong>on</strong> the identified risk scores or priority. M<strong>on</strong>itoring<br />

changes in risk scores over time and the use of methodologies and tools supporting management of<br />

risks (e.g. Nati<strong>on</strong>al Institute of Standards and Technology methodology, OCTAVE, COBRA etc.)<br />

closes the loop <strong>on</strong> RM processes, enabling c<strong>on</strong>tinual m<strong>on</strong>itoring of the effectiveness of RM<br />

approaches (Elky, 2006). However, effective RM approaches al<strong>on</strong>e is not sufficient. <strong>Management</strong> and<br />

stakeholder support and buy-in, development and enforcement of policies that deal with new and<br />

emerging risks, and development of a risk culture that involves training and communicati<strong>on</strong> of RM<br />

activities are also essential. Further, IT RM should not exist in a silo; many authors highlight the<br />

importance of integrating IT RM approaches into the overall Enterprise Risk <strong>Management</strong> (ERM)<br />

framework - an approach that involves holistically managing the enterprises entire risk portfolio<br />

(Fraser and Simkins, 2010; Kouns and Minoli, 2010). As RM seeks to protect the organizati<strong>on</strong>s assets<br />

and missi<strong>on</strong>, it needs to be regarded as a management functi<strong>on</strong> as opposed to merely a technical<br />

activity (Elky, 2006). Integrating IT risk with ERM practices promotes a greater understanding by IT of<br />

the business priorities and protecti<strong>on</strong> of critical business services, and enables more effective risk<br />

mitigati<strong>on</strong>, avoidance of risk oversights and better return <strong>on</strong> IT investments (Silic<strong>on</strong> Republic, 2010).<br />

The following secti<strong>on</strong>s of this paper c<strong>on</strong>siders the maturity modeling approach to improving risk<br />

management capabilities, by enabling organizati<strong>on</strong>s to understand their current RM capabilities and<br />

identify practices to improve their capability maturity overtime. The maturity modeling approach has<br />

been well adopted in IS research. Secti<strong>on</strong> 3 provides a brief overview of maturity models prior to<br />

introducing a new capability maturity framework that addresses organizati<strong>on</strong>s RM capability maturity.<br />

3. Maturity models in IS research<br />

Maturity models are “c<strong>on</strong>ceptual models that outline anticipated, typical, logical and desired evoluti<strong>on</strong><br />

paths towards maturity” (Becker et al, 2010), where maturity is “a measure to evaluate the capabilities<br />

of an organizati<strong>on</strong> in regards to a certain discipline” (Rosemann and de Bruin, 2005). Maturity can<br />

also be regarded as “an evoluti<strong>on</strong>ary progress in the dem<strong>on</strong>strati<strong>on</strong> of a specific ability or in the<br />

accomplishment of a target from an initial to a desired or normally occurring end stage” (Mettler,<br />

2009). Maturity models outline characteristics associated with various levels of maturity, thereby<br />

serving as the basis for an organizati<strong>on</strong>’s capability maturity assessment. In essence, they serve to<br />

help organizati<strong>on</strong>s to understand their “as is” situati<strong>on</strong> and enable them to transiti<strong>on</strong> to the desired “to<br />

be” maturity, through deriving and implementing specific practices or improvement roadmaps. These<br />

improvement maps support a stepped progressi<strong>on</strong> with respect to organizati<strong>on</strong>s capabilities, enabling<br />

them to fulfill the characteristics required to meet specific maturity levels.<br />

A recent literature review of maturity models in IS research has highlighted a growing interest in this<br />

area (Becker et al, 2010; Mettler, 2009), in order to inform organizati<strong>on</strong>al c<strong>on</strong>tinuous improvements<br />

and support either self or third party maturity assessments. While the Software Engineering Institute’s<br />

(SEI) Capability Maturity Model (CMM) for software development and the successor Capability<br />

Maturity Model Integrati<strong>on</strong> (CMMI) are most prevalent in studies of maturity (Becker et al, 2010),<br />

n<strong>on</strong>etheless, several new maturity models have been developed in recent years. These focus <strong>on</strong><br />

improving maturity in, for example, IT/business alignment (Luftman, 2003; Khaiata and Zualkernan,<br />

2009); business process management (Rosemann and de Bruin, 2005); business intelligence<br />

(Hewlett Packard, 2007); project management (Crawford, 2006); informati<strong>on</strong> lifecycle management<br />

(Sun, 2005); digital government (Gottschalk, 2009); inter-organizati<strong>on</strong>al systems adopti<strong>on</strong> (Ali et al,<br />

2011) and enterprise resource planning systems use (Holland and Light, 2001).<br />

35


Marian Carcary<br />

Despite the growing interest in this area, according to Becker et al (2010), IS research has “rarely<br />

endeavored into reflecting and developing theoretically sound maturity models” and as such there is a<br />

lack of evidence of scientifically rigorous methods in their development processes, with some models<br />

based <strong>on</strong> poor theoretical foundati<strong>on</strong>s (Mettler, 2009). Methods, such as Design Science (DS)<br />

(Hevner et al, 2004) are proposed as a useful means to develop new maturity models in a rigorous<br />

manner, using both prior studies and empirical evidence as the basis for the model’s c<strong>on</strong>tent<br />

development and stages of maturity. Further, Becker et al (2010) suggests that there is a lack of<br />

evidence of validity testing of newly developed models; however to ensure their relevance for<br />

practiti<strong>on</strong>ers, the proposed models need to be piloted and “applicability checks” c<strong>on</strong>ducted with<br />

practiti<strong>on</strong>ers. Closing the gap between current and desired maturity is also problematic, with Mettler<br />

(2009) suggesting that many models do not describe how to carry out improvements acti<strong>on</strong>s.<br />

In line with the categorizati<strong>on</strong> of maturity models adopted by Becker et al (2010) (prescriptive,<br />

descriptive, descriptive/prescriptive, descriptive/reflective, and reflective), this paper reflects a<br />

prescriptive c<strong>on</strong>tributi<strong>on</strong> (i.e. a specificati<strong>on</strong> of how capability improvements could take place) through<br />

the presentati<strong>on</strong> of a new maturity model. The model presented addresses the c<strong>on</strong>cerns outlined<br />

above through following a rigorous development process based <strong>on</strong> design science and open<br />

innovati<strong>on</strong> principles; empirical piloting, testing and validati<strong>on</strong> of the model; and development of a<br />

series of improvement practices, outcomes and metrics to drive maturity level progressi<strong>on</strong>.<br />

4. Presenting a new maturity model - the IT capability maturity framework (IT<br />

CMF)<br />

The IT CMF (Figure 1) is a capability maturity model developed at the Innovati<strong>on</strong> Value Institute (IVI),<br />

Nati<strong>on</strong>al University of Ireland Maynooth. It represents a systematic framework to enable CIO’s/CEO’s<br />

to understand and improve their organizati<strong>on</strong>’s maturity in order to derive business value from IT<br />

investments (Curley, 2004; 2007). The framework represents an emerging blueprint of IT capabilities<br />

and serves as an assessment tool which enables organizati<strong>on</strong>s to understand and improve over time<br />

their IT capability across five levels of maturity. At a macro level, the IT CMF c<strong>on</strong>sists of four<br />

integrated IT management strategies (macro capabilities); these comprise 33 critical capabilities<br />

(CCs) which represent key activities of the IT organizati<strong>on</strong>.<br />

Figure 1: IT CMF (source: Innovati<strong>on</strong> Value Institute)<br />

C<strong>on</strong>tent development for the IT CMF is undertaken by the IVI c<strong>on</strong>sortium. The c<strong>on</strong>sortium is made up<br />

of over 80 industry partners linked to IVI through a comm<strong>on</strong> desire to develop and enhance their<br />

organizati<strong>on</strong>’s understanding of improved business value through IT capability management. The<br />

c<strong>on</strong>sortium members are invited and encouraged to participate in the research and development<br />

activities of the IVI through workgroup c<strong>on</strong>tributi<strong>on</strong>. A work group exists for each of the 33 CCs, which<br />

include a mix of Subject Matters Experts (SMEs) and Key Opini<strong>on</strong> Leaders (KOLs), including<br />

36


Marian Carcary<br />

academic researchers, industry-based practiti<strong>on</strong>ers, and c<strong>on</strong>sultants. Work group development output<br />

evolves through a series of four stages and is reviewed at the end of each stage by a technical<br />

committee (TC). As development work progresses through the various stages, more in-depth c<strong>on</strong>tent<br />

is required and the CC material is subject to more rigorous reviews and validati<strong>on</strong> processes.<br />

This c<strong>on</strong>tent development across the four stages follows the Design Science (DS) research approach.<br />

This approach is increasingly recognised within IS as an important complement to the prevalent<br />

behavioral science. DS is a problem solving approach that involves building and evaluating innovative<br />

artifacts in a rigorous manner to solve complex, real world, relevant problems, make research<br />

c<strong>on</strong>tributi<strong>on</strong>s that extend the boundaries of what is already known, and communicate the results to<br />

appropriate audiences (Gregor and J<strong>on</strong>es, 2007; Hevner et al, 2004; March and Smith, 1995; March<br />

and Storey, 2008; Pries-Heje and Baskerville, 2008; Purao, 2002; Venable, 2006). Knowledge and<br />

understanding of the problem domain is achieved through artifact c<strong>on</strong>structi<strong>on</strong> and evaluati<strong>on</strong> (Hevner<br />

et al, 2004). The DS approach adopted in the IT CMF development (Table 1) is closely aligned with<br />

the three DS research cycles proposed by Hevner (2007). (For a detailed discussi<strong>on</strong> of its<br />

development, see Carcary (2011)).<br />

Table 1: DS Cycles of the IT CMF development<br />

DS Cycle IT CMF<br />

DS Relevance Cycle Relevance of the IT CMF artifact is driven by the problems organizati<strong>on</strong>s<br />

experience in optimizing how they currently manage and measure the business<br />

value of their IT investments. Field testing of the IT CMF in the applicati<strong>on</strong><br />

envir<strong>on</strong>ment helps determine if further development work is required to ensure<br />

its relevance in addressing the business problem.<br />

DS Rigor Cycle Development is grounded in existing artifacts, methodologies, foundati<strong>on</strong>al<br />

theories and expertise and draws from an extensive base of industry and<br />

academic literature and existing IT standards and frameworks. C<strong>on</strong>tributi<strong>on</strong>s to<br />

the knowledge base include a detailed framework and set of practices that help<br />

drive innovati<strong>on</strong> and change in how organizati<strong>on</strong>s manage and use their IT<br />

investments to optimize business value.<br />

DS Design Cycle Development focuses <strong>on</strong> iterative build and evaluate activities by the CC<br />

workgroup to address the identified problem, while drawing <strong>on</strong> existing<br />

theoretical foundati<strong>on</strong>s and methodologies in the knowledge base. The build<br />

process is evolved and refined through evaluati<strong>on</strong> feedback, including technical<br />

committee stage gate reviews to identify further development refinements and<br />

field testing of the artifact within c<strong>on</strong>textually diverse organizati<strong>on</strong>s.<br />

4.1 An examinati<strong>on</strong> of the risk management critical capability<br />

Located within the IT-CMF’s “Managing IT like a business” macro capability, the Risk <strong>Management</strong><br />

CC focuses <strong>on</strong> proactively assessing, prioritizing, handling and m<strong>on</strong>itoring risks in order to minimize<br />

exposure to and the potential impact of IT risk. This CC aims to be holistic in addressing the key<br />

categories of IT risk facing organizati<strong>on</strong>s, including for example IT security; data protecti<strong>on</strong> and<br />

informati<strong>on</strong> privacy; operati<strong>on</strong>s/ business c<strong>on</strong>tinuity and disaster recovery; IT investment; IT<br />

programme, project and product life cycles; IT service c<strong>on</strong>tracts and suppliers; IT image/ brand; IT<br />

pers<strong>on</strong>nel; regulatory/ legal and ethics policy compliance, as well as emerging risks in these and other<br />

categories. The assessment provides key insights into an organizati<strong>on</strong>s maturity with respect to three<br />

key areas - governance, risk profile design and the actual risk management processes. These three<br />

categories are comprised of nine capability building blocks (CBBs), as outlined in Table 2.<br />

Table 2: Capability building blocks of the RM CC<br />

Governance Profile Design Process<br />

Policies for Risk <strong>Management</strong> Definiti<strong>on</strong> of risk profiles Risk assessment<br />

Integrati<strong>on</strong> into IT leadership and<br />

Risk prioritizati<strong>on</strong><br />

governance structures<br />

<strong>Management</strong>, governance and<br />

Risk handling<br />

performance management<br />

Communicati<strong>on</strong>s and training Risk m<strong>on</strong>itoring<br />

The above nine CBBs are the focus areas of a RM assessment, with dedicated maturity questi<strong>on</strong>s<br />

developed within each of these areas. Examples of RM maturity assessment questi<strong>on</strong> topics are<br />

outlined in Table 3.<br />

37


Marian Carcary<br />

Table 3: Example RM maturity assessment questi<strong>on</strong> topics<br />

Key areas of the IT CMF RM Maturity Assessment<br />

Definiti<strong>on</strong> and implementati<strong>on</strong> of risk policies;<br />

Establishing risk policies ownership and resp<strong>on</strong>sibilities;<br />

Integrating RM into IT leadership and governance structures;<br />

Identifying RM roles and resp<strong>on</strong>sibilities;<br />

Identifying levels of senior management support;<br />

Measuring the effectiveness and efficiency of RM activities;<br />

Training stakeholders in RM;<br />

Disseminating RM policies, processes and results;<br />

Determining collaborati<strong>on</strong> levels between risks managers;<br />

Defining risk profiles by their potential impact;<br />

Using risk profiles in risk assessment and mitigati<strong>on</strong>;<br />

Identifying subject matter experts for risk assessments;<br />

Identifying and scoring risks and their impact;<br />

Prioritizing risks and risk handling strategies;<br />

Identifying tools to support risk handling;<br />

Assigning ownership to identified risks;<br />

Defining and implementing appropriate risk c<strong>on</strong>trols;<br />

M<strong>on</strong>itoring and reporting identified risks and the effectiveness of risk c<strong>on</strong>trols.<br />

Assessment questi<strong>on</strong>s in these and other areas describe maturity level statements that follow IT CMF<br />

prescribed maturity level logic, across five stages – initial, basic, intermediate, advanced and<br />

optimized. Maturity assessment participants are invited to score the organizati<strong>on</strong>’s maturity across<br />

these five levels, as well as identify the future desired state. Aggregated scores support reporting of<br />

the organizati<strong>on</strong>’s self-assessed current and desired maturity levels; in additi<strong>on</strong> an IVI assessment,<br />

based <strong>on</strong> both the survey assessment results and in-depth interviews with key RM stakeholders result<br />

in a formal IVI maturity assessment score and presentati<strong>on</strong> of a set of practices to support the<br />

organizati<strong>on</strong> transiti<strong>on</strong>ing to higher maturity levels. A detailed set of IVI RM Practices, Outcomes, and<br />

Metrics (POMs) at the various maturity stages support closing the gap between organizati<strong>on</strong>s’ current<br />

and desired maturity states.<br />

As such, the RM maturity assessment represents the basis for organizati<strong>on</strong>s understanding their key<br />

strengths and weaknesses in their ability to mitigate potential IT risks. The output from the IT CMF RM<br />

assessment enables an organizati<strong>on</strong> to put acti<strong>on</strong> plans in place to mature their capability in<br />

effectively managing IT risks <strong>on</strong> the horiz<strong>on</strong>. In general, transiti<strong>on</strong>ing to higher maturity levels requires<br />

an organizati<strong>on</strong> to align and integrate business objectives with RM practices; define and implement<br />

effective processes for risk assessment, prioritizati<strong>on</strong>, handling and mitigati<strong>on</strong> for all risk areas,<br />

including new and emerging risks, and integrate them into enterprise RM processes; create an<br />

effective and integrated risk register; obtain support from senior management; ensure l<strong>on</strong>g-term<br />

training and retenti<strong>on</strong> of skills; and embed RM into IT and business activities. Adopting these and<br />

other practices in order to mature RM capabilities and proactively manage risks becomes an<br />

important step in deriving business value from IT investments.<br />

The following secti<strong>on</strong> provides an analysis of the value of this maturity model and the maturity model<br />

approach in supporting the transiti<strong>on</strong> to higher maturity levels and more effective IT capabilities.<br />

5. Discussi<strong>on</strong> and c<strong>on</strong>clusi<strong>on</strong>s<br />

Growth in the development and use of maturity models provides str<strong>on</strong>g support for the relevance of<br />

the maturity assessment approach in practice. As stated by Mettler (2009), “as organizati<strong>on</strong>s<br />

c<strong>on</strong>stantly face the pressures to obtain and retain competitive advantage, invent and reinvent new<br />

products and services, reduce cost and time to market, and enhance quality at the same time, the<br />

need for and the development of new maturity models will certainly not diminish given that they assist<br />

decisi<strong>on</strong> makers to balance these sometimes divergent objectives <strong>on</strong> a more or less comprehensive<br />

manner”. Based <strong>on</strong> the literature, the greatest c<strong>on</strong>cern regarding this assessment approach is the<br />

processes involved in maturity model development – rather than building <strong>on</strong> a theoretical basis, many<br />

models are simply based <strong>on</strong> practices drawn from organizati<strong>on</strong> or industry specific projects that<br />

dem<strong>on</strong>strated favourable results, for many models there is a lack of model testing in terms of validity,<br />

reliability and generalizability, and little documentati<strong>on</strong> <strong>on</strong> how the model was designed and<br />

developed (Mettler, 2009).<br />

38


Marian Carcary<br />

Based <strong>on</strong> the above, it can be suggested that given the relevance of maturity models to organizati<strong>on</strong>s<br />

in informing and supporting prioritized stepped improvements in capabilities, a maturity model that<br />

addresses the c<strong>on</strong>cerns in the literature pertaining to their theoretical foundati<strong>on</strong>s and rigorous<br />

development and testing approaches should be a useful c<strong>on</strong>tributi<strong>on</strong>. The framework for IT<br />

management outlined in this paper, and more specifically for maturing the RM capability, therefore<br />

should reflect an important c<strong>on</strong>tributi<strong>on</strong> from the perspective of organizati<strong>on</strong>s seeking to optimize their<br />

RM capabilities and the value they derive from IT. Through adopting the maturity modelling approach<br />

to RM and improving maturity overtime, it is proposed that CEOs and CIOs can improve the<br />

organizati<strong>on</strong>’s ability to manage risks and protect the business from risk impacts; they can reduce the<br />

organizati<strong>on</strong>’s exposure to risks such as IT security, IT sabotage, data protecti<strong>on</strong> and informati<strong>on</strong><br />

privacy, and IT investment risks; they can increase the likelihood of meeting the scope, cost, time and<br />

quality targets of projects by effectively managing associated IT risks; they can increase the<br />

likelihood of compliance with external regulati<strong>on</strong>s and ethics policies; and they can increase<br />

transparency of how IT risks map/ relate to business objectives and decisi<strong>on</strong>s. In essence,<br />

organizati<strong>on</strong>s with a mature RM capability are more effective in proactively managing IT risks, and in<br />

reducing the exposure to and the potential impact of IT risks.<br />

As outlined above, the presentati<strong>on</strong> of the IT-CMF’s risk management critical capability is a<br />

prescriptive c<strong>on</strong>tributi<strong>on</strong>; further research is needed to investigate the extent to which this maturity<br />

model supports capability maturity progressi<strong>on</strong> in a real world setting over time. As such, future<br />

research will involve a series of multiple case studies <strong>on</strong> a l<strong>on</strong>gitudinal basis to determine the realworld<br />

value of this approach in improving organizati<strong>on</strong>s capabilities in managing existing, and new<br />

and emerging risks.<br />

References<br />

Ali, M., Kurnia, S., Johnst<strong>on</strong>, R. (2011). Understanding the Progressive Nature of Inter-Organizati<strong>on</strong>al Systems<br />

(IOS) Adopti<strong>on</strong>. E-Collaborati<strong>on</strong> Technologies and Organizati<strong>on</strong>al Performance: Current and Future Trends,<br />

Ed. 1, pp. 124-144.<br />

Amberg, M. and Okujava, S. (2005). State of the art of IT project value analysis. In (Ed. D. Remenyi),<br />

Proceedings of the 12 th <str<strong>on</strong>g>European</str<strong>on</strong>g> <str<strong>on</strong>g>C<strong>on</strong>ference</str<strong>on</strong>g> <strong>on</strong> Informati<strong>on</strong> Technology Evaluati<strong>on</strong>, pp. 21-34. Turku,<br />

Finland, 29 th -30 th September, <strong>Academic</strong> <str<strong>on</strong>g>C<strong>on</strong>ference</str<strong>on</strong>g>s, Reading.<br />

Becker, J., Niehaves, B., Poppelbus, J., and Sim<strong>on</strong>s, A. (2010). Maturity Models in IS Research. Proceedings of<br />

the 18 th <str<strong>on</strong>g>European</str<strong>on</strong>g> <str<strong>on</strong>g>C<strong>on</strong>ference</str<strong>on</strong>g> in Informati<strong>on</strong> System, pp 1-12. Available at:<br />

http://web.up.ac.za/ecis/ECIS2010PR/ECIS2010/C<strong>on</strong>tent/Papers/0320.pdf<br />

Benaroch, M., Lichtenstein, Y. and Robins<strong>on</strong>, K. (2006). Real opti<strong>on</strong>s in Informati<strong>on</strong> Technology risk<br />

management: an empirical validati<strong>on</strong> of real-opti<strong>on</strong> relati<strong>on</strong>ships. MIS Quarterly, 30,(4), 827-864.<br />

Brown, A. (2005). IS evaluati<strong>on</strong> in practice. In (Ed. D. Remenyi), Proceedings of the 12 th <str<strong>on</strong>g>European</str<strong>on</strong>g> <str<strong>on</strong>g>C<strong>on</strong>ference</str<strong>on</strong>g><br />

<strong>on</strong> Informati<strong>on</strong> Technology Evaluati<strong>on</strong>, pp. 109-118. Turku, Finland, 29 th -30 th September, <strong>Academic</strong><br />

<str<strong>on</strong>g>C<strong>on</strong>ference</str<strong>on</strong>g>s, Reading.<br />

Carcary, M. (2011) “Design Science Research: The Case of the IT Capability Maturity Framework (IT<br />

CMF)” Electr<strong>on</strong>ic Journal of Business Research Methods, 9,(2), p109-118.<br />

Casey, T. (2007). Threat Agent Library Helps Identify Informati<strong>on</strong> Security Risks, Intel White Paper<br />

Curley, M. (2004). Managing informati<strong>on</strong> Technology for business value – practical strategies for IT and business<br />

managers. Intel Press.<br />

Curley, M. (2007). Introducing an IT Capability Maturity Framework, Internati<strong>on</strong>al <str<strong>on</strong>g>C<strong>on</strong>ference</str<strong>on</strong>g> <strong>on</strong> Enterprise<br />

Informati<strong>on</strong> Systems.<br />

Crawford, J.K. (2006). The project management maturity model. Informati<strong>on</strong> Systems <strong>Management</strong>, 23,(4), pp.<br />

50-58.<br />

Elky, S. (2006). An Introducti<strong>on</strong> to informati<strong>on</strong> systems risk management. SANS Institute. Available at:<br />

http://www.sans.org/reading_room/whitepapers/auditing/introducti<strong>on</strong>-informati<strong>on</strong>-system-riskmanagement_1204<br />

Ernst and Young, (2011). Global Informati<strong>on</strong> Security Survey. Available at:<br />

http://www.ey.com/GL/en/Services/Advisory/2011-Global-Informati<strong>on</strong>-Security-Survey---Plugging-the-dataleaks<br />

Fraser, J. and Simkins, B. (2010). Enterprise Risk <strong>Management</strong>: Today's Leading Research and Best Practices<br />

for Tomorrow's Executives, Wiley<br />

Frost and Sullivan (2011). The 2011 (ISC) 2 Global Informati<strong>on</strong> Security Workforce Study. Available at:<br />

https://www.isc2.org/uploadedFiles/Landing_Pages/NO_form/2011GISWS.pdf.<br />

Gaudin, S. (2007). TK Maxx security breach costs soar to 10 times earlier estimate. Informati<strong>on</strong><br />

Week. August 15, 2007.<br />

Glass, R. (2006). Looking into the challenges of complex IT projects. Communicati<strong>on</strong>s of the ACM, 49,(11), 15-<br />

17.<br />

39


Marian Carcary<br />

Gottschalk, P. (2009). Maturity levels for interoperability in digital government. Government Informati<strong>on</strong> Quarterly,<br />

26 (1), pp75-81.<br />

Gregor, S. and J<strong>on</strong>es, D. (2007). The anatomy of a design theory. Journal of the Associati<strong>on</strong> of<br />

Informati<strong>on</strong> Systems, 8,(5), 312-335.<br />

Hevner, A., March, S. and Park, J. (2004). Design Science in Informati<strong>on</strong> Systems research. MIS<br />

Quarterly. 28,(1), 75-105.<br />

Hewlett Packard, (2007). The HP Business intelligence maturity model. Available at:<br />

http://h20195.www2.hp.com/v2/GetDocument.aspx?docname=4AA1-5467ENW&cc=us&lc=en<br />

Holland, C.P. and Light, B. (2001). A stage maturity model for enterprise resource planning systems use.<br />

Database for Advances in Informati<strong>on</strong> Systems, 32(2), pp 24-45.<br />

ISACA (2010). ISACA US IT risk/reward barometer survey. Available at: http://www.isaca.org/About-<br />

ISACA/Press-room/News-Releases/2010/Pages/ISACA-US-IT-Risk-Reward-Barometer-Survey.aspx<br />

Khaiata, M. and Zualkernan, I.A. (2009). A simple instrument to measure IT business alignment maturity.<br />

Informati<strong>on</strong> Systems <strong>Management</strong>, 26(2), pp 138-152.<br />

Kirk, J. (2007). Est<strong>on</strong>ia recovers from massive denial-of-service attack. NetworkWorld. May 17, 2007.<br />

Kouns, J. and Minoli, D. (2010). Informati<strong>on</strong> Technology Risk <strong>Management</strong> in Enterprise Envir<strong>on</strong>ments, Wiley<br />

Luftman, J. (2003). Assessing IT-Business Alignment. Informati<strong>on</strong> Systems <strong>Management</strong>. 20(4) pp 9-15.<br />

March, S. and Smith, G. (1995) Design and natural science research <strong>on</strong> informati<strong>on</strong> technology, Decisi<strong>on</strong><br />

Support Systems 15,(4), 251–266.<br />

March, S.T. and Storey, V.C. (2008). Design Science in the Informati<strong>on</strong> Systems discipline: an introducti<strong>on</strong> to the<br />

special issue <strong>on</strong> design science research, MIS Quarterly, 32,(4), 725-730.<br />

Mettler, T. (2009). A design science research perspective <strong>on</strong> maturity models in Informati<strong>on</strong> Systems. St. Gallen:<br />

Institute of Informati<strong>on</strong> <strong>Management</strong>, Universtiy of St. Gallen.<br />

Pries-Heje, J. and Baskerville, R. (2008). The Design Theory Nexus, MIS Quarterly, 32,(4), 731-755.<br />

Purao, S. (2002). “Design Research in the Technology of Informati<strong>on</strong> Systems: Truth or Dare.” Georgia State<br />

University, Department of CIS Working Paper. Atlanta.<br />

Rosemann, M. and de Bruin,T. (2005). Towards a business process management maturity model. In<br />

Proceedings of the <str<strong>on</strong>g>European</str<strong>on</strong>g> <str<strong>on</strong>g>C<strong>on</strong>ference</str<strong>on</strong>g> <strong>on</strong> Informati<strong>on</strong> Systems, Regenburg, Germany.<br />

Rosenquist, M. (2007). Measuring the Return <strong>on</strong> IT Security Investments, Intel White Paper.<br />

Silic<strong>on</strong> Republic (2010). Closing the gaps between ICT and enterprise risk management. Available at:<br />

http://www.silic<strong>on</strong>republic.com/news/item/16901-closing-the-gaps-between-ic<br />

St<strong>on</strong>eburner, G., Goguen, A. and Feringa, A. (2002). Risk <strong>Management</strong> guide for Informati<strong>on</strong> Technology<br />

systems. Nati<strong>on</strong>al Institute of Standards and Technology. US Department of Commerce.<br />

Sun, (2005). Informati<strong>on</strong> lifecycle management maturity model. Available at:<br />

http://www.dynasys.com/Downloads/Sun_ILM_Maturity_Model_2005.pdf<br />

Taylor, H. (2006). Critical risks in outsourced IT projects: the intractable and the unforeseen. Communicati<strong>on</strong>s of<br />

the ACM, 49,(11), 75-79.<br />

Venable, J.R. (2006). The role of theory and theorizing in design science research. In Proceedings of the First<br />

Internati<strong>on</strong>al <str<strong>on</strong>g>C<strong>on</strong>ference</str<strong>on</strong>g> <strong>on</strong> Design Science Research in Informati<strong>on</strong> Systems. 24 th -25 th February,<br />

Clarem<strong>on</strong>t, CA.<br />

Westerman, G. and Hunter, R. (2007). IT Risks. Harvard Business School Publishing<br />

40


IS Evaluati<strong>on</strong> in the Fusi<strong>on</strong> View: An Emergence<br />

Perspective<br />

Sven Carlss<strong>on</strong> and Olgerta T<strong>on</strong>a<br />

Informatics, Lund University School of Ec<strong>on</strong>omics and <strong>Management</strong>, Lund,<br />

Sweden<br />

Sven.Carlss<strong>on</strong>@ics.lu.se<br />

Olgerta.T<strong>on</strong>a@ics.lu.se<br />

Abstract: Different theories have changed the way we address to technology. In the debate <strong>on</strong> the core of<br />

Informati<strong>on</strong> System (IS), El Sawy identified three faces of IS views: c<strong>on</strong>necti<strong>on</strong>, immersi<strong>on</strong>, and fusi<strong>on</strong>. El Sawy<br />

c<strong>on</strong>tends that it may be time for a natural shift of emphasis from the c<strong>on</strong>necti<strong>on</strong> view to the immersi<strong>on</strong> view to the<br />

fusi<strong>on</strong> view as IT c<strong>on</strong>tinues to morph and augment its capabilities. In the fusi<strong>on</strong> view, IT and IS are fused within<br />

the business envir<strong>on</strong>ment, such that business and IT and IS are indistinguishable to standard time-space<br />

percepti<strong>on</strong> and form a unified fabric. There exist different traditi<strong>on</strong>al IS evaluati<strong>on</strong> approaches, like experimental,<br />

pragmatic, c<strong>on</strong>structivist, pluralist and realist IS evaluati<strong>on</strong> research. These approaches evaluate IS when it is<br />

positi<strong>on</strong>ed either in the c<strong>on</strong>necti<strong>on</strong> or immersi<strong>on</strong> view. However, we believe that the fusi<strong>on</strong> view will influence the<br />

way IS is evaluated. This paper uses the relati<strong>on</strong>al emergence theory, based <strong>on</strong> the philosophy of critical realism<br />

where emergence refers to an entity as a ‘whole’ whose parts are structured by the relati<strong>on</strong>s am<strong>on</strong>g each other.<br />

Emergent entities possess properties different from its individual parts’ properties. C<strong>on</strong>sidering the similarity to<br />

fusi<strong>on</strong> c<strong>on</strong>cept, the emergence c<strong>on</strong>cept is used to theorize and operati<strong>on</strong>alize the fusi<strong>on</strong> view, as it lacks a<br />

theoretical grounding. Based <strong>on</strong> this, we present and discuss the implicati<strong>on</strong>s for IS evaluati<strong>on</strong> in terms of how to<br />

evaluate a process as well as the output of the process. The discussi<strong>on</strong> <strong>on</strong> IS evaluati<strong>on</strong> is illustrated through an<br />

empirical example, drawn <strong>on</strong> a research study within a police organizati<strong>on</strong>. This paper c<strong>on</strong>cludes that in the<br />

fusi<strong>on</strong> view, the evaluati<strong>on</strong> process shall embrace a holistic perspective. The focus of the evaluati<strong>on</strong> process<br />

shall be the emergent entity which c<strong>on</strong>sist of IS, users, task and processes structured by means of relati<strong>on</strong>ships<br />

am<strong>on</strong>g each other. The properties exhibited by this emergent entity shall be evaluated.<br />

Keywords: IS evaluati<strong>on</strong>, fusi<strong>on</strong> view, emergence<br />

1. Introducti<strong>on</strong><br />

The development and expansi<strong>on</strong> of evaluati<strong>on</strong> theory and practice is at the core of several different<br />

disciplines. It is important to scrutinize theories, approaches, and models used in evaluati<strong>on</strong><br />

(research) as well as evaluati<strong>on</strong> research approaches’ philosophical underpinnings (Carlss<strong>on</strong>, 2003).<br />

Recently, different theories have changed the way we address to technology. Orlikowski (2010) says<br />

that this type of research can be characterized with the label ‘entanglement in practice’. She says that<br />

influential entanglement perspectives are Actor Network Theory and the noti<strong>on</strong> of sociomateriality. On<br />

the other hand, El Sawy (2003) presented three different views <strong>on</strong> IS: c<strong>on</strong>necti<strong>on</strong>, immersi<strong>on</strong>, and<br />

fusi<strong>on</strong>. He c<strong>on</strong>tends that it may be time for a natural shift of emphasis from the c<strong>on</strong>necti<strong>on</strong> view to the<br />

immersi<strong>on</strong> view to the fusi<strong>on</strong> view as IT c<strong>on</strong>tinues to morph and augment its capabilities. In the<br />

c<strong>on</strong>necti<strong>on</strong> view, IT and IS are viewed as separable artefacts and artificial systems that are used by<br />

people as tools. They are separable from work, processes, and people. In the immersi<strong>on</strong> view, IT and<br />

IS are immersed as part of the business envir<strong>on</strong>ment and cannot be separated from work, processes,<br />

and the systemic properties of intra- and inter-organizati<strong>on</strong>al processes and relati<strong>on</strong>ships. This view<br />

stresses work c<strong>on</strong>text and systemic relati<strong>on</strong>ships and mutual interdependencies. In the fusi<strong>on</strong> view, IT<br />

and IS are fused within the business envir<strong>on</strong>ment, such that business and IT and IS are<br />

indistinguishable to standard time-space percepti<strong>on</strong> and form a unified fabric. Hence, IT-enabled work<br />

and processes are treated as <strong>on</strong>e.<br />

Most of the IS evaluati<strong>on</strong> approaches, such as experimental, pragmatic, c<strong>on</strong>structivist, pluralist and<br />

realist <strong>on</strong>es, are approaches used within the IS c<strong>on</strong>necti<strong>on</strong> and immersi<strong>on</strong> view. We believe that the<br />

fusi<strong>on</strong> view will influence the way IS is evaluated and the already existing approaches have<br />

drawbacks if used in fusi<strong>on</strong> view.<br />

We argue that IS evaluati<strong>on</strong> research based <strong>on</strong> the principles and philosophy of critical realism will<br />

push forward the “traditi<strong>on</strong>al” IS evaluati<strong>on</strong> research approaches. In critical realism there is a<br />

tendency to see structure as “…the c<strong>on</strong>stituent comp<strong>on</strong>ents [of structure and agency] cannot be<br />

examined separately....In the absence of any degree of aut<strong>on</strong>omy it becomes impossible to examine<br />

their interplay” (Archer 1988). The relati<strong>on</strong>al emergence theory, based <strong>on</strong> the philosophy of critical<br />

41


Sven Carlss<strong>on</strong> and Olgerta T<strong>on</strong>a<br />

realism (Elder – Vass, 2010), introduces the emergence c<strong>on</strong>cept which is similar to the fusi<strong>on</strong><br />

c<strong>on</strong>cept. Both fusi<strong>on</strong> and the emergence c<strong>on</strong>cept refer to different parts coming together by<br />

interacting and acting as <strong>on</strong>e identity. If the parts are split up, the same entity with the same<br />

properties will no l<strong>on</strong>ger be obtained.<br />

This paper has two main c<strong>on</strong>tributi<strong>on</strong>s. The first is theoretical, in which fusi<strong>on</strong> view introduced by El<br />

Sawy (2003) will be discussed and elaborated <strong>on</strong> a theoretical level. The sec<strong>on</strong>d c<strong>on</strong>tributi<strong>on</strong> will be a<br />

discussi<strong>on</strong> <strong>on</strong> the implicati<strong>on</strong>s of IS evaluati<strong>on</strong> in the fusi<strong>on</strong> view, in terms of how to evaluate a<br />

process as well as the output of the process. The discussi<strong>on</strong> will be illustrated by means of an<br />

empirical example.<br />

The remainder of the paper is organized as follows. The next secti<strong>on</strong> presents a brief summary of the<br />

main IS evaluati<strong>on</strong> approaches and the corresp<strong>on</strong>ding views where they are used. Secti<strong>on</strong> 3 will<br />

discuss the emergence c<strong>on</strong>cept and its properties which are going to be adopted in the study. This is<br />

followed by an empirical example. C<strong>on</strong>clusi<strong>on</strong>s are presented in the final secti<strong>on</strong>.<br />

2. IS evaluati<strong>on</strong> approaches<br />

This secti<strong>on</strong> briefly reviews the IS evaluati<strong>on</strong> research approaches based <strong>on</strong> Carlss<strong>on</strong> (2003). Table 1<br />

presents a short descripti<strong>on</strong> of each of the following approaches: experimental, pragmatic,<br />

c<strong>on</strong>structivist, pluralist and realistic evaluati<strong>on</strong> approach. Further these approaches are classified<br />

based <strong>on</strong> which IS view (El Sawy, 2003) they are more appropriate to be used.<br />

Table 1: IS evaluati<strong>on</strong> approaches<br />

IS evaluati<strong>on</strong><br />

approach<br />

Experimental<br />

Pragmatic<br />

C<strong>on</strong>structivist<br />

Pluralist<br />

Realistic<br />

Descripti<strong>on</strong> Views<br />

Builds <strong>on</strong> the logic of experimentati<strong>on</strong>: take two more or less<br />

matched groups (situati<strong>on</strong>s) and treat <strong>on</strong>e group and not the<br />

other. By measuring both groups before and after the<br />

treatment of the <strong>on</strong>e, an evaluator can get a “clear” measure of<br />

the impact of the treatment.<br />

Represents a use-led model of evaluati<strong>on</strong> research, stressing<br />

utilizati<strong>on</strong>: the basic aim of IS evaluati<strong>on</strong> research is to develop<br />

IS initiatives (implementati<strong>on</strong> of IS) which solve “problems”—<br />

problems can be organizati<strong>on</strong>al problems like reduced<br />

competitiveness or far from good customer services.<br />

IS initiatives should not be treated “…as ‘independent<br />

variables’, as ‘things’, as ‘treatments’, as ‘dosages’.” (Paws<strong>on</strong><br />

and Tilley, 1997). Instead all IS initiatives are “…c<strong>on</strong>stituted in<br />

complex processes of understanding and interacti<strong>on</strong>” and an<br />

IS initiative (IS implementati<strong>on</strong>) will work “through a process of<br />

reas<strong>on</strong>ing, change, influence, negotiati<strong>on</strong>, battle of wills,<br />

persuasi<strong>on</strong>, choice increase (or decrease), arbitrati<strong>on</strong> or some<br />

such like.” (Paws<strong>on</strong> and Tilley, 1997).<br />

Combines the strengths of the three approaches: an approach<br />

combining the rigor of experimentati<strong>on</strong> with the practice of<br />

pragmatism, and with the c<strong>on</strong>structivist’s empathy for the<br />

voices of the stakeholders.<br />

Its aim to produce ever more detailed answers to the questi<strong>on</strong><br />

of why an IS initiative—IS, types of IS, or IS implementati<strong>on</strong>—<br />

works for whom and in what circumstances.<br />

C<strong>on</strong>necti<strong>on</strong><br />

C<strong>on</strong>necti<strong>on</strong><br />

C<strong>on</strong>necti<strong>on</strong>,<br />

Immersi<strong>on</strong><br />

C<strong>on</strong>necti<strong>on</strong>,<br />

Immersi<strong>on</strong><br />

C<strong>on</strong>necti<strong>on</strong>,<br />

Immersi<strong>on</strong><br />

In general, the approaches listed in Table 1 can be used in the two IS views described by El Sawy<br />

(2003), which are the c<strong>on</strong>necti<strong>on</strong> and immersi<strong>on</strong> view. All these approaches c<strong>on</strong>sider IS during the<br />

evaluati<strong>on</strong> either as a tool, separated from the work and process –in the case of the c<strong>on</strong>necti<strong>on</strong> view,<br />

or as immersed in the work and process – in the case of the immersi<strong>on</strong> view. N<strong>on</strong>e of them c<strong>on</strong>sider<br />

the evaluati<strong>on</strong> of IS, when it is fused in the organizati<strong>on</strong> and as such IS is not treated as <strong>on</strong>e together<br />

with the task, processes and people.<br />

Taking in c<strong>on</strong>siderati<strong>on</strong> the emergence of IS towards fusi<strong>on</strong> view, there is a need for the evaluati<strong>on</strong><br />

methods to be revised. IS can no l<strong>on</strong>ger be separated from the business envir<strong>on</strong>ment, as it is already<br />

fused in it. During the evaluati<strong>on</strong>, IS has to be c<strong>on</strong>sidered as a whole, together with the users, the<br />

tasks and the processes.<br />

42


3. Emergence in critical realism<br />

Sven Carlss<strong>on</strong> and Olgerta T<strong>on</strong>a<br />

Critical realism has become an important perspective in modern philosophy and social science<br />

(Archer et al, 1998, Robs<strong>on</strong> 2002), but critical realism is to a large extent absent in IS research. We<br />

argue that IS evaluati<strong>on</strong> research based <strong>on</strong> the principles and philosophy of critical realism<br />

overcomes some of the problems associated with “traditi<strong>on</strong>al” IS evaluati<strong>on</strong> research approaches.<br />

Its manifesto is to recognize the reality of the natural order and the events and discourses of the<br />

social world. It holds that “we will <strong>on</strong>ly be able to understand—and so change—the social world if we<br />

identify the structures at work that generate those events and discourses … These structures are not<br />

sp<strong>on</strong>taneously apparent in the observable pattern of events; they can <strong>on</strong>ly be identified through the<br />

practical and theoretical work of the social sciences.” (Bhaskar, 1989).<br />

Elder–Vass (2010) has introduced the relati<strong>on</strong>al emergence theory based <strong>on</strong> the philosophy of critical<br />

realism. He provides a general <strong>on</strong>tological framework to discuss the social structures and human<br />

individuals as entities with emergent properties which determine the social events. An entity is a<br />

‘whole’, which c<strong>on</strong>sists of parts structured by means of the relati<strong>on</strong>s am<strong>on</strong>g each-other. Emergent<br />

entities possess some properties produced by mechanisms which depend <strong>on</strong> the properties of<br />

individual parts and the way the parts are structured in order to form the entity (whole). The properties<br />

which derive from the entity are not possessed by its individual parts. The way the parts are related at<br />

a certain point in time will depict the joint effect they will have. Therefore the relati<strong>on</strong> between the<br />

entity and its parts is not of causati<strong>on</strong>, but of compositi<strong>on</strong> (Elder-Vass, 2010).<br />

The importance of the interacti<strong>on</strong>s between the parts is expressed by Holland (1998, pp. 121-122) as<br />

below:<br />

Emergence is above all a product of coupled, c<strong>on</strong>text-dependent interacti<strong>on</strong>s.<br />

Technically these interacti<strong>on</strong>s, and the resulting system, are n<strong>on</strong>linear: The behavior of<br />

the overall system cannot be obtained by summing the behaviors of its c<strong>on</strong>stituent<br />

parts… the whole is indeed more than the sum of its parts. However, we can reduce the<br />

behavior of the whole to the lawful behavior of its parts, if we take the n<strong>on</strong>linear<br />

interacti<strong>on</strong>s into account.<br />

There are some elements which an emergent entity should own (Elder-Vass, 2010). First of all, the<br />

different parts which an emergent entity c<strong>on</strong>sists of should be recognized. The relati<strong>on</strong>ships between<br />

the parts which cause this type of entity should be identified. The emergent entity should be explained<br />

in terms of morphogenetic and morphostatic causes. Morphogenetic refers to “those processes which<br />

tend to elaborate or change a system’s given form, structure or state” (Buckley, 1967, p.58-59).<br />

Morphostasis refers to the causes which maintain an entity either internal (the causes which maintain<br />

the parts in a certain relati<strong>on</strong>ship) or external (the causes coming from the envir<strong>on</strong>ment).To the later,<br />

Buckley (1967, p.58-9) defines as “those processes in complex system-envir<strong>on</strong>ment exchanges that<br />

tend to preserve or maintain a system’s given form, organizati<strong>on</strong> or state”.<br />

Additi<strong>on</strong>ally, De Wolf and Holvoet (2005), based <strong>on</strong> a literature review, have listed different properties<br />

possessed by an emergent entity:<br />

Interactive Parts. The interacti<strong>on</strong> between the parts is resp<strong>on</strong>sible for the emergent system (Odell,<br />

2002; Heylighen, 2002).<br />

Micro-Macro level effect. The emergent properties that an emergent system shows as the results<br />

of the interacti<strong>on</strong> between its comp<strong>on</strong>ents (Holland, 1998).<br />

Novelty. Emergent properties cannot be understood by the properties of the comp<strong>on</strong>ents.<br />

Anyhow, they still can be studied via the comp<strong>on</strong>ents and their relati<strong>on</strong>s in the c<strong>on</strong>text of the<br />

whole system (Holland, 1998; Elder-Vass, 2010).<br />

Coherence. The emergence property tends to maintain its own identity during the time, by<br />

c<strong>on</strong>verting the interactive parts into a ‘whole’ (Heylighen, 2002).<br />

Dynamical. Emergence properties of a system are related to the time dimensi<strong>on</strong>, so that they can<br />

arise or change over time (Holland, 1998).<br />

Decentralised C<strong>on</strong>trol. No parts al<strong>on</strong>e can direct or c<strong>on</strong>trol the emergent properties of a system<br />

(Odell, 2002).<br />

43


Sven Carlss<strong>on</strong> and Olgerta T<strong>on</strong>a<br />

Two-Way Link. The interacti<strong>on</strong> between the parts influences the emergent system, which <strong>on</strong> the<br />

other hand can influence its individual parts. (Odell, 2002)<br />

Flexibility. As no single part is fully resp<strong>on</strong>sible for the emergent properties of a system, its<br />

substituti<strong>on</strong> or n<strong>on</strong>-functi<strong>on</strong>ality will not lead to a total failure of the emergent entity (Odell, 2002).<br />

To summarize: In the last view proposed by El Sawy (2003), fusi<strong>on</strong> view, IT and IS, are fused within<br />

the business envir<strong>on</strong>ment and are indistinguishable. They can no l<strong>on</strong>ger be separated from work,<br />

processes and users, but instead they shall be treated as <strong>on</strong>e. Both IS fusi<strong>on</strong> and the emergent entity<br />

refer to different elements merging together by interacting and acting as <strong>on</strong>e identity. If the parts are<br />

split up, no l<strong>on</strong>ger will the same entity with the same properties be obtained. Using emergence as a<br />

c<strong>on</strong>ceptual lens we argue that in the fusi<strong>on</strong> view different parts such as IS, tasks and users,<br />

structured by relati<strong>on</strong>s am<strong>on</strong>g each other, give rise to an emerging entity showing different emerging<br />

properties. In this sense we try to operati<strong>on</strong>alize and theorize fusi<strong>on</strong> further more.<br />

Additi<strong>on</strong>ally, as already menti<strong>on</strong>ed in the previous secti<strong>on</strong>, IS fusi<strong>on</strong> view calls for IS evaluati<strong>on</strong><br />

approaches to be revised. We argue that IS evaluati<strong>on</strong> research based <strong>on</strong> the emergence theory will<br />

move the evaluati<strong>on</strong> from the c<strong>on</strong>necti<strong>on</strong> and immersi<strong>on</strong> view, towards the fusi<strong>on</strong> view. IS can no<br />

l<strong>on</strong>ger be evaluated separated from its users, processes and tasks. The evaluati<strong>on</strong> process should<br />

take a holistic perspective. The parts which c<strong>on</strong>stitute the whole and their relati<strong>on</strong>s shall be<br />

recognized in order to understand the generati<strong>on</strong> of the impacts and events. Hence, the impacts of<br />

the entity as a whole shall be evaluated, shifting the focus from the IS impact to the ‘whole’ impact<br />

where IS is part of the entity. When evaluating the impacts and benefits of IS the macro – level<br />

impacts should be c<strong>on</strong>sidered. It means that the emergent properties need to be evaluated instead<br />

and the mechanisms which bring the emergent properties shall be described. In this way, based <strong>on</strong><br />

the evaluati<strong>on</strong> results, changes in different parts can be undertaken if necessary to maintain the<br />

whole as such and keep IS fused in the organizati<strong>on</strong>.<br />

4. Empirical example<br />

In this secti<strong>on</strong> we will illustrate via an empirical example how to begin an IS evaluati<strong>on</strong> based <strong>on</strong> the<br />

emergence theory. The empirical example is based <strong>on</strong> a l<strong>on</strong>gitudinal research study we have been<br />

performing in a police organizati<strong>on</strong> since 2009. For more details about the case, see Carlss<strong>on</strong> et al<br />

(2010). Skåne is the third largest police authority in Sweden and it has approximately 3240<br />

employees, where approximately 2340 are police officers and 900 civil servants. The BI, created with<br />

the software QlikView, started as a single applicati<strong>on</strong> based <strong>on</strong> the system RAR (a system for crime<br />

statistics, where all reported crimes are registered). (QlikView is a BI/DSS software company, see:<br />

http://www.qlikview.com). The system was used by crime analysts to forecast when and where crime<br />

could occur. The system creates associati<strong>on</strong>s <strong>on</strong> the processed data which makes it easy to<br />

distinguish relati<strong>on</strong>ships between them. The informati<strong>on</strong> can be visualized by diagrams, tables or<br />

dashboards. We argue that the police organizati<strong>on</strong>s’ BI system is slowly emerging to the fusi<strong>on</strong> view.<br />

We will evaluate the societal impact of an entity c<strong>on</strong>sisting of: users, BI system and the specific task<br />

to be solved with two different examples. The first example is a single-shot analysis for solving a<br />

crime. The sec<strong>on</strong>d example is an <strong>on</strong>-going analysis of crimes for improving crime preventi<strong>on</strong>.<br />

Case 1: Finding the serial shooter. BI usage in the police organizati<strong>on</strong> in Skåne proved to be<br />

successful in the soluti<strong>on</strong> of a crime which was scaring the citizens of Malmö, a city situated in the<br />

southern part of Sweden. During the last years, a serial shooter in Malmö shot many people<br />

(emigrants, sec<strong>on</strong>d generati<strong>on</strong> emigrants, and refugees) in the streets, at bus stops, and in their cars.<br />

Many were seriously wounded and <strong>on</strong>e was killed. For this case, after finding a suspect, the police in<br />

Skåne gathered all the reports dating back to 1998 and found that there were about 58 reports<br />

c<strong>on</strong>nected to him. They used a BI applicati<strong>on</strong> which can read about 1.5 milli<strong>on</strong> reports in less than 10<br />

minutes. According to <strong>on</strong>e of our interviewees the specific applicati<strong>on</strong> took about “four hours to<br />

build—because we knew how to do it”. During the analysis 6–7 key words were used in the free-text<br />

search applicati<strong>on</strong>. The analysis produced “27000–32000 rows [of informati<strong>on</strong>] in Excel with 11–13<br />

words in every row”. This informati<strong>on</strong> identified the reports that should be read and evaluated.<br />

Reading and evaluating had to be d<strong>on</strong>e the old way (manually).<br />

Case 2: Crime preventi<strong>on</strong>. One example of ‘car theft’ preventi<strong>on</strong> in Malmö shows how hot spot<br />

analyses can be both effective and efficient in terms of crime preventi<strong>on</strong>. According to Weisburd and<br />

Telep (2010), crime hot spot strategies for fighting crime have been embraced recently by some<br />

police forces. The idea underpinning the crime hot spot strategy is that crime is better prevented by<br />

44


Sven Carlss<strong>on</strong> and Olgerta T<strong>on</strong>a<br />

focusing <strong>on</strong> areas (hot spots), for example, specific streets, buildings, blocks, and areas within a<br />

community or z<strong>on</strong>e, rather than by focusing <strong>on</strong> individuals. The process started by using the BI<br />

system to point out some areas with the highest number of car thefts. Further analyses, by means of<br />

other systems, proceeded to identify the parking lots and streets of those hot z<strong>on</strong>es. Acti<strong>on</strong> was taken<br />

to allocate the patrol forces to the hot spots. This strategy revealed to be effective in relati<strong>on</strong> to<br />

reducing crime not <strong>on</strong>ly in the hot spot areas but also in most z<strong>on</strong>es of the city. Thus, the BI system<br />

enabled the police to implement the hot spot strategy.<br />

The successful Malmö-case within the police organizati<strong>on</strong> indicates the BI usage in crime soluti<strong>on</strong><br />

where the case against the shooter will be str<strong>on</strong>ger in the court. In this case the interacti<strong>on</strong> between<br />

the three parts: BI system, users and the task to be solved led to macro-level effects. Based <strong>on</strong> the<br />

type of organizati<strong>on</strong> under study, we refer to these effects as societal benefits. The users c<strong>on</strong>structed<br />

the necessary applicati<strong>on</strong> within BI to read the reports, saving m<strong>on</strong>ths of work. In this case, time is<br />

critical as the police have to handle a serial shooter and the so<strong>on</strong>er he is caught, the better. Hence,<br />

we deduce that the interacti<strong>on</strong> between the parts resulted in time reducti<strong>on</strong> (enhanced efficiency) and<br />

str<strong>on</strong>gly supported the crime soluti<strong>on</strong>.<br />

In additi<strong>on</strong> to the Malmö- case, the same analyses can be applied to the ‘car theft’ example. As a<br />

result of the interacti<strong>on</strong> between BI system, the users and the task to be solved, societal benefits in<br />

terms of crime preventi<strong>on</strong> are produced. The entity –BI, users and task, in terms of emergence, has<br />

brought novel ways of dealing with crime preventi<strong>on</strong>, hot spot, which cannot be achieved by these<br />

parts operating <strong>on</strong> their own. Although hot spot analyses have been criticized for moving the crime to<br />

the “next corner”, many other studies and experiments have c<strong>on</strong>cluded that hot spot analyses are<br />

followed by the diffusi<strong>on</strong> of crime preventi<strong>on</strong> benefits (Weisburd and Telep, 2010). This study supports<br />

the arguments of Weisburd and Telep (2010) because the benefits of crime preventi<strong>on</strong> were not<br />

limited to the hot spots but also extended to the nearby areas. Now the police can identify hot spots<br />

and better allocate their resources with the main intenti<strong>on</strong> of preventing crime. The interacti<strong>on</strong>s<br />

between BI and users to solve different tasks has improved efficiency in terms of reducing the number<br />

of target spots and allocating resources better, leading to improved crime preventi<strong>on</strong>. Hence, we<br />

observe some sort of emerging entity giving rise to macro effects by means of the interacting parts,<br />

which cannot produce these results <strong>on</strong> their own in isolati<strong>on</strong>. C<strong>on</strong>sidering this process in terms of<br />

morphogenic causes, the interacti<strong>on</strong> between BI and the users for a specific task, the capabilities of<br />

the BI system to analyse data very fast, and the ability of the employees to interpret the data are the<br />

main c<strong>on</strong>tributory causes which give rise to the exhibiti<strong>on</strong>, by the emerging entity, of macro-level<br />

effects which cannot be reached <strong>on</strong> a micro-level.<br />

On the other hand, it is worth c<strong>on</strong>sidering that in both examples the interacti<strong>on</strong> between BI and users<br />

existed <strong>on</strong>ly during the first phases of the crime analyses, where all the data needed are collected.<br />

Afterwards, the users had to carry out manual work: for instance in the Malmö case they had to<br />

analyse the reports manually and in the case of ‘car theft’ they had to use other sources to obtain<br />

more details regarding the specific ‘hot’ streets and blocks. This means that we find emergent<br />

properties in the first phases of analytical tasks but not during the other phases, because of a lack of<br />

interacti<strong>on</strong> between the users and BI. In this case we cannot identify the morphostatic causes<br />

(internal) which keep stable an emergent system. Hence, as the morphostatic causes are missing, we<br />

can deduce that there exists no stable emergent entity. However, we believe that l<strong>on</strong>ger interacti<strong>on</strong><br />

and extra capabilities of BI to support all of the phases of analytical work will drive BI towards the total<br />

fusi<strong>on</strong> view. This study support the arguments of El Sawy (2003) that more technological advances in<br />

BI will shift it to fusi<strong>on</strong> view and at the same time we will observe more solid emergent structures.<br />

In coherence terms, in the case of the serial shooter some identity properties are shown. The news<br />

headline “Swedish Police Arrest Man over Malmö Racist Shootings” (Associated Press in Malmo,<br />

2010) dem<strong>on</strong>strates how neither the analyst group nor the BI system whose interacti<strong>on</strong> saved so<br />

many m<strong>on</strong>ths of work were menti<strong>on</strong>ed, but the entity was given another name: the Swedish police.<br />

The entity is recognized by the organizati<strong>on</strong>’s name, as the interacti<strong>on</strong> between the BI and users to<br />

solve the crime takes place within the c<strong>on</strong>text of the police organizati<strong>on</strong>. One questi<strong>on</strong> in this case is:<br />

will the macro-level effects drive micro-level behaviour in leading to a two-way link? Basically, the<br />

success of this interacti<strong>on</strong> between the users and the BI in crime soluti<strong>on</strong> and preventi<strong>on</strong> may lead<br />

the users to a more extensive usage of the system in other cases, where their needs will also drive<br />

updating of the BI system or inclusi<strong>on</strong> of other technological capabilities in it. For instance, during the<br />

‘hot spot’ analysis, the users realized that the integrati<strong>on</strong> of a map in their BI system would further<br />

45


Sven Carlss<strong>on</strong> and Olgerta T<strong>on</strong>a<br />

improve hot spot analyses, and would, as stated by a resp<strong>on</strong>dent, “take our work very, very far”. In its<br />

current state BI displays the hot crime z<strong>on</strong>es by means of z<strong>on</strong>e numbers, but it is unable to direct the<br />

police to specific streets or buildings referred to as spots. Additi<strong>on</strong>ally, we can observe the<br />

characteristic of flexibility. If we substitute the users with others, nearly the same effects and benefits<br />

will be obtained, and also if the BI system is down some of the work can be d<strong>on</strong>e manually, but that<br />

would result in a waste of time, and in some cases time can be critical, for instance in crime<br />

preventi<strong>on</strong> or soluti<strong>on</strong>. For example in the case of the Malmo serial shooter, if the BI system went<br />

down, the users could still manage but the work would take about 9 m<strong>on</strong>ths, which could even lead to<br />

the release of the suspect until the evidence was ready. To summarize, this was an empirical example<br />

where the evaluati<strong>on</strong> of societal impact was focused <strong>on</strong> the emergent properties of the entity user – BI<br />

- tasks. To achieve a complete emergence to fusi<strong>on</strong> view, technological improvements are required to<br />

be implemented in the BI system in order for it to be fully - fused with the analysts’ tasks too during<br />

the whole phases of analyses.<br />

5. C<strong>on</strong>clusi<strong>on</strong>s<br />

The existing IS evaluati<strong>on</strong> approaches fall into IS c<strong>on</strong>necti<strong>on</strong> view or immersi<strong>on</strong> view. The fusi<strong>on</strong> view,<br />

calls for the IS evaluati<strong>on</strong> approaches to be revised. This paper uses the relati<strong>on</strong>al emergence theory<br />

based <strong>on</strong> critical realism to theorize and operati<strong>on</strong>alize further the fusi<strong>on</strong> view. Based <strong>on</strong> the<br />

emergence c<strong>on</strong>cept, within IS fusi<strong>on</strong> view an emerging entity rise, where IS is a part c<strong>on</strong>stituting this<br />

entity. This paper c<strong>on</strong>tinued further <strong>on</strong> discussing the implicati<strong>on</strong>s of the evaluati<strong>on</strong> process when IS<br />

is positi<strong>on</strong>ed in fusi<strong>on</strong> view. We suggest for the evaluati<strong>on</strong> process to embrace a ‘holistic’ perspective,<br />

where IS, users, task and processes should be c<strong>on</strong>sidered as <strong>on</strong>e entity. The relati<strong>on</strong>ship between<br />

the parts and the entities’ properties should be evaluated. Therefore emergence is used as a<br />

c<strong>on</strong>ceptual lens and we illustrated the idea by evaluating the organizati<strong>on</strong>al and societal impact of the<br />

BI system in a police organizati<strong>on</strong> in a holistic perspective. The arguments of El Sawy (2003) that with<br />

more technological advances fusi<strong>on</strong> view will be reached are supported in the example. Once more<br />

capabilities will be implemented to BI, it will be extensively used and other emergent properties may<br />

result from it.<br />

References<br />

Archer, M. (1988) Culture and Agency: The Place of Culture in Social Theory, Cambridge University Press,<br />

Cambridge, UK.<br />

Archer, M., Bhaskar, R., Collier, A., Laws<strong>on</strong>, T. and Norrie, A. (Eds) (1998) Critical Realism: Essential Readings,<br />

Routledge, L<strong>on</strong>d<strong>on</strong>.<br />

Associated Press in Malmo (2010) “Swedish police arrest man over Malmö racist shootings”, The Guardian,<br />

Available At : < http://www.guardian.co.uk/world/2010/nov/07/malmo-race-shooting-arrest>, Accessed 13<br />

February 2012.<br />

Bhaskar, R. (1989) Reclaiming Reality, Verso, L<strong>on</strong>d<strong>on</strong>.<br />

Buckley. W. (1967) Sociology and modern systems theory, Englewood Cliffs, NJ: Prentice Hall.<br />

Carlss<strong>on</strong>, S.A. (2003) “Advancing Informati<strong>on</strong> Systems Evaluati<strong>on</strong> (Research): A Critical Realist<br />

Approach”, Electr<strong>on</strong>ic Journal of Informati<strong>on</strong> Systems Evaluati<strong>on</strong>, Vol 6, No. 2, pp. 11-20.<br />

Carlss<strong>on</strong>, S., Skog, L.-M. and T<strong>on</strong>a, O. (2010) An IS success evaluati<strong>on</strong> of a DSS in a police organizati<strong>on</strong>. In A.<br />

Respício, F. Adam, G. Phillips-Wren, C. Teixeira, and J. Telhada Eds. Bridging the socio-technical gap in<br />

decisi<strong>on</strong> support systems, IOS Press, Amsterdam, , pp. 443–454.<br />

De Wolf, T. and Holvoet, T. (2005) Emergence versus self-organisati<strong>on</strong>: Different c<strong>on</strong>cepts but promising when<br />

combined. In Brueckner, S., Di Marzo Serugendo, G., Karageorgos, A., Nagpal, R., Eds. Engineering Self<br />

Organising Systems: Methodologies and Applicati<strong>on</strong>s. Lecture Notes in Computer Science, Springer Verlag:<br />

Berlin, pp. 1–15.<br />

Elder-Vass D. (2010) The Causal Power of Social Structures: Emergence, Structure and Agency, Cambridge<br />

University Press, New York.<br />

El Sawy, O. A. (2003) “The IS Core IX: The 3 Faces of IS Identity: C<strong>on</strong>necti<strong>on</strong>, Immersi<strong>on</strong>, and Fusi<strong>on</strong>”,<br />

Communicati<strong>on</strong>s of the AIS, Vol 12, pp. 588-598.<br />

Heyligen, F. (2002) The science of self-organisati<strong>on</strong> and adaptivity. In: The Encyclopedia of Life Support<br />

Systems. UNESCO Publishing-Eolss Publishers.<br />

Holland J.H. (1998) Emergence: From chaos to order, Oxford: Oxford University Press.<br />

Odell, J. (2002) “Agents and complex systems”, JOT, Vol 1, pp. 35–45.<br />

Orlikowski, W.J. (2010) “The sociomateriality of organisati<strong>on</strong>al life: c<strong>on</strong>sidering technology in management<br />

research”, Cambridge Journal of Ec<strong>on</strong>omics, Vol 34, pp 125-141.<br />

Paws<strong>on</strong>, R. and N. Tilley (1997) Realistic Evaluati<strong>on</strong>, Sage, L<strong>on</strong>d<strong>on</strong>.<br />

Robs<strong>on</strong>, C. (2002) Real World Research, Sec<strong>on</strong>d editi<strong>on</strong>, Blackwell, Oxford.<br />

Weisburd, D. and Telep, C. (2010) “The efficiency of place based policing”, Journal of Police Studies Vol 17, pp.<br />

247-262.<br />

46


Where do Tablets fit in the Organizati<strong>on</strong>’s Workstati<strong>on</strong><br />

Inventory?<br />

Mitch Cochran 1 and Paul Witman 2<br />

1 City of M<strong>on</strong>rovia, USA<br />

2 California Lutheran University, USA<br />

mcochran@ci.m<strong>on</strong>rovia.ca.us<br />

pwitman@callutheran.edu<br />

Abstract: Tablets have become the new technology of choice for end users by end users. They are excited<br />

about the new technology and have the feeling that the tablets can replace other computing devices. Marketing<br />

studies dem<strong>on</strong>strate the significant user demand for tablets. Users are now questi<strong>on</strong>ing organizati<strong>on</strong>s’ IT<br />

operati<strong>on</strong>s as to why they can’t be used in daily business. It is the job of the IT operati<strong>on</strong> to guide the<br />

incorporati<strong>on</strong> of devices into the organizati<strong>on</strong>s computing device inventory. A critical issue is for the organizati<strong>on</strong><br />

to dem<strong>on</strong>strate what ‘apps’ are appropriate and what business applicati<strong>on</strong>s require a traditi<strong>on</strong>al platform. This<br />

paper will examine the strengths, weaknesses, and issues of incorporating tablets into the organizati<strong>on</strong>, to help<br />

put structure around the problem for organizati<strong>on</strong>s and their IT staffs, and to define a future research agenda.<br />

Tablets have some of the same attributes and limitati<strong>on</strong>s of both cell ph<strong>on</strong>es and workstati<strong>on</strong>s. Tablets have<br />

str<strong>on</strong>g capabilities for c<strong>on</strong>tent retrieval where more powerful laptops are better suited for c<strong>on</strong>tent creati<strong>on</strong>. Tablets<br />

provide easy access to Internet-based applicati<strong>on</strong>s. Using the Internet use paradigm tablets can use touch or pen<br />

based input for easy navigati<strong>on</strong>. But developing c<strong>on</strong>tent is more than easy navigati<strong>on</strong> around websites or using<br />

pre-determined ‘apps’. When users need more complex input tasks such as coding with advanced tools, they<br />

need the support of a laptop or workstati<strong>on</strong>. Some functi<strong>on</strong>s just need additi<strong>on</strong>al screen space that tablets d<strong>on</strong>’t<br />

offer. Organizati<strong>on</strong>s need to understand how the new technology format differs from cell ph<strong>on</strong>es, traditi<strong>on</strong>al<br />

workstati<strong>on</strong>s or laptops so governance procedures can be modified to incorporate appropriate c<strong>on</strong>trols. The<br />

paper explores various user business functi<strong>on</strong>s and the device types that best fit them. The paper then discusses<br />

issues regarding hardware differences, security and applicati<strong>on</strong> development. The organizati<strong>on</strong> will need to<br />

maintain security procedures similar to that needed of a laptop <strong>on</strong> topics such as antivirus, data leakage, and<br />

remote access. The organizati<strong>on</strong> will treat it similarly to a cell ph<strong>on</strong>e in governance <strong>on</strong> issues such as employeeprovided<br />

technology or remote deleti<strong>on</strong> capabilities. From a governance perspective, the distincti<strong>on</strong>s am<strong>on</strong>g<br />

workstati<strong>on</strong>s, tablets and ph<strong>on</strong>es are less clear. We propose a research agenda to move the study of this topic<br />

forward.<br />

Keywords: governance, tablets, workstati<strong>on</strong>s<br />

1. Introducti<strong>on</strong><br />

Organizati<strong>on</strong>s need to be c<strong>on</strong>stantly evaluating new technologies against their business and user<br />

needs. Tablets have appeared <strong>on</strong> the scene as the newest device with the ‘WOW’ factor. The users<br />

have become enamored with the tablet for pers<strong>on</strong>al use as an easy to use device. The general public<br />

has asked the questi<strong>on</strong> of why it can’t be used at work. The IT operati<strong>on</strong>s groups need to understand<br />

where how to make the users happy by supporting tablets but also c<strong>on</strong>trol them to ensure that they fit<br />

business goals and governance.<br />

Industry surveys have shown that tablets are quickly making inroads into the technology inventory.<br />

Guy Currier of Baseline Magazine posted that 11% of resp<strong>on</strong>dents expected a very str<strong>on</strong>g increase in<br />

the tablet investment for 2011 [10]. 14% of the resp<strong>on</strong>dents expected a str<strong>on</strong>g increase and 25%<br />

expected a moderate increase. So 50% of the resp<strong>on</strong>dents expected an increase in tablet<br />

acquisiti<strong>on</strong>s. The expected increase was higher that other current technologies such as virtualizati<strong>on</strong><br />

(44%), Business intelligence (35%) or Social communicati<strong>on</strong>s (35%).<br />

Similarly, the CIO Insight survey [8] shows that 35% of companies have deployed tablets, 31% are<br />

testing tablets, 26% are evaluating or tracking them, and 7% have not expressed an interest. The<br />

survey defined tablets as Apple iPad, Blackberry Playbook or similar devices. The survey also found<br />

the tablets would provide greater productivity or cost savings for 40% of the resp<strong>on</strong>dents, improve<br />

business agility or versatility for 50% of the resp<strong>on</strong>dents and open up new market opportunities for<br />

43% of the resp<strong>on</strong>dents.<br />

The same CIO Insight survey shows that 46% of companies have deployed smart ph<strong>on</strong>es as mobile<br />

clients. The survey asked how the smart ph<strong>on</strong>es addressed business goals. 28% of the resp<strong>on</strong>dents<br />

felt that the smart ph<strong>on</strong>e was best for cost-savings or productivity goals, 47% of the resp<strong>on</strong>dents felt<br />

47


Mitch Cochran and Paul Witman<br />

that smart ph<strong>on</strong>es were best at improving business versatility, and 39% of the resp<strong>on</strong>dents felt that<br />

smart ph<strong>on</strong>es were best for opening up new markets or opportunities.<br />

The survey shows that there is a positive interest but the industry does not have a clear picture of how<br />

tablets will be used. The Dimensi<strong>on</strong>al Research study of May 2011 showed that 72% of the tablets<br />

had not been formally deployed. [9] The survey showed 41% are used by individuals who have<br />

purchased <strong>on</strong> their own (a phenomen<strong>on</strong> known popularly as “Bring your own device”, or “BYOD”). The<br />

survey asked for the reas<strong>on</strong> for the iPad selecti<strong>on</strong> and found that 53% resp<strong>on</strong>ded favorably due to the<br />

availability of productivity tools. 35% of the resp<strong>on</strong>dents felt that the “cool” factor was driving the<br />

business demand. The survey also showed that 51% of the resp<strong>on</strong>dents did not have a clearly<br />

articulated strategy for adopting tablets. A critical piece of any hardware is the software that runs <strong>on</strong> it.<br />

Many users d<strong>on</strong>’t understand what software runs well <strong>on</strong> a tablet or smart ph<strong>on</strong>e as compared to what<br />

runs well <strong>on</strong> a traditi<strong>on</strong>al laptop. The Dimensi<strong>on</strong>al study showed that 42% of the resp<strong>on</strong>dents found<br />

that the need for applicati<strong>on</strong> development is not understood [9]. The resp<strong>on</strong>dents believed that any<br />

applicati<strong>on</strong> could run <strong>on</strong> the tablet.<br />

2. Current applicati<strong>on</strong>s and tasks<br />

Gartner lists the most comm<strong>on</strong> tablet business applicati<strong>on</strong>s or tasks as: [12]<br />

Pers<strong>on</strong>al Office Automati<strong>on</strong> (documents and spreadsheets)<br />

Presentati<strong>on</strong><br />

Note Taking<br />

Task <strong>Management</strong><br />

File <strong>Management</strong><br />

Dictati<strong>on</strong><br />

IT Admin Utilities<br />

Forrester found in a tablet usage survey that 75% of owners purchased the device to complement<br />

other devices instead of being a replacement device [13]. They also found that after the purchase of<br />

the tablet, users spent 47% percent less time <strong>on</strong> their computers. Dan Blacharski cites a<br />

Lenovo/Qualcomm summary that found the major uses for c<strong>on</strong>sumer or business envir<strong>on</strong>ments [14]<br />

which is listed in table 1.<br />

Table 1: C<strong>on</strong>sumers and business uses for tablets<br />

C<strong>on</strong>sumer Use Business Use<br />

Gaming 84% Browsing 73%<br />

Browsing 78% Email 69%<br />

Email 74% Working Remotely 67%<br />

Sales Support 46%<br />

3. Device characteristics<br />

Customer Representative 45%<br />

Informati<strong>on</strong> technology managers need to understand where the product fits into the workplace. The<br />

tool has benefits and limitati<strong>on</strong>s which are listed in table 2:<br />

Table 2: Tablet and laptop characteristics<br />

Tablet Laptop<br />

Cost Higher for equal machines Less cost, includes keyboard, and<br />

other Items that are extra in a tablet<br />

Power Less compute power More processing power, more<br />

memory<br />

Storage Limited Larger but unlimited due to USB<br />

ports<br />

Mouse No track point / tract pad, Not as accurate Includes internal and external<br />

mouse, Can be very accurate<br />

Graphics Limited High resoluti<strong>on</strong><br />

Keyboard On screen takes space Extra, may be flat Included<br />

Media Some with USB No CD / DVD USB is standard Includes CD / DVD<br />

Multitasking Not yet Included in operating system<br />

48


4. Hardware<br />

Mitch Cochran and Paul Witman<br />

Tablets are too new in the marketplace to be able to accurately differentiate the various products<br />

based <strong>on</strong> reliability. From our limited experience and lack of notoriety of any particular device, tablets<br />

seem to have the same basic reliability as traditi<strong>on</strong>al laptops. We have found no discussi<strong>on</strong> in the<br />

trade journals that compare reliability. A tablet has limited processing power as compared to a laptop.<br />

Over time, the processing capacity should increase due to advances in technology but currently,<br />

laptops have faster processors, as well as greater memory and storage. Some tablets d<strong>on</strong>’t have an<br />

integrated USB or FireWire ports. Users may need USB ports to provide for c<strong>on</strong>nectivity to external<br />

keyboards, CD or DVD drives, memory keys and or thumb drives. The tablet design philosophy<br />

seems to focus <strong>on</strong> c<strong>on</strong>nectivity to the ‘Cloud’. Some manufacturers have allowed for keyboards to be<br />

attached using Bluetooth technologies.<br />

Tablets have an integrated touch keyboard. There are a couple of issues that limit keyboard<br />

performance. The developers have worked hard <strong>on</strong> making an effective the key recogniti<strong>on</strong>. The<br />

issue is whether the operator gets tactile feedback when the keys are struck. For touch typists, normal<br />

keyboards have a raised butt<strong>on</strong> <strong>on</strong> the ‘F’ and ‘J’ keys for people to recognize that their fingers are in<br />

the correct positi<strong>on</strong>. You will not find the raised bumps <strong>on</strong> a touch screen. The tablets can have<br />

attached keyboards. Some of these are flat keyboards that d<strong>on</strong>’t provide the same touch or tactile<br />

feedback that people expect in a laptop. There is also an issue of using a touch keyboard that it uses<br />

half of the screen therefore, the operator has less screen to look at. The touch keyboard erg<strong>on</strong>omics<br />

are also an issue. The laptop may be put <strong>on</strong> a flat service. This positi<strong>on</strong> will put the hands in an<br />

incorrect typing positi<strong>on</strong> according to erg<strong>on</strong>omic standards. The slate may also have a glare due to<br />

the reflecti<strong>on</strong> of the screen since it is now laying flat. The glare could make it difficult to type <strong>on</strong>. A<br />

laptop can adjust the viewing angle to pers<strong>on</strong>al preference. From a developer’s point of view, the<br />

tablet keyboard will be limited in the availability of characters. The keyboard <strong>on</strong> the iPad has the ‘’ keys (used in html coding) <strong>on</strong> the third keyboard template. Some users have menti<strong>on</strong>ed that it<br />

has been hard to fix mistakes. The mouse or touch features may make it more challenging to get to<br />

an exact point depending <strong>on</strong> the f<strong>on</strong>t size within a document.<br />

Both tablets and laptops can take advantage of data input bey<strong>on</strong>d that of just a mouse or keyboard.<br />

Ant Ozok et all discuss the advantages of using a digital pen and or handwriting recogniti<strong>on</strong> [7]. They<br />

pointed out that handwriting recogniti<strong>on</strong> accuracy is highly correlated to acceptance and use of the<br />

recogniti<strong>on</strong> tools. The more that the tool is used, the higher the level of acceptance becomes. The<br />

digital pen allows for the ease of handwriting data entry. Microsoft has provided their Notes software<br />

for Windows-based laptops. It dem<strong>on</strong>strates the ease of use for a handwriting recogniti<strong>on</strong> interface<br />

that can be used in the business envir<strong>on</strong>ment.<br />

Battery life may be a c<strong>on</strong>cern to some buyers where mosttablets do not allow for the changing of<br />

batteries. An advantage for laptops has been the capability to carry a spare charged battery for l<strong>on</strong>ger<br />

trips or time away from power. People can argue that it is not an issue since they have not had a<br />

problem with l<strong>on</strong>g term battery life. Tablets have the advantage of less power use due to smaller<br />

display screens and more efficient comp<strong>on</strong>ents such as solid state disk drives. Tablets have not been<br />

around l<strong>on</strong>g enough to determine if fixed batteries or battery life are an issue.<br />

5. Video and graphics<br />

The tablet has specialized graphics for a relatively small touch screen. Laptops can have larger<br />

screens with higher resoluti<strong>on</strong>s. The laptops can also support multiple graphics adaptors allowing for<br />

more graphics capabilities.<br />

Both laptops and tablets have added cameras as standard devices. The issue for organizati<strong>on</strong>s is<br />

how to manage the photos that are taken. The current tablets and laptops provide resoluti<strong>on</strong> similar to<br />

stand al<strong>on</strong>e cameras. One of the key downfalls of tablet cameras is that they provide limited zoom<br />

capabilities. Stand al<strong>on</strong>e cameras still offer more capabilities for organizati<strong>on</strong>s to document situati<strong>on</strong>s.<br />

The issue for organizati<strong>on</strong>s is still how to manage records that are derived from pictures.<br />

The marketplace has not determined a single standard for video c<strong>on</strong>ferencing. Applicati<strong>on</strong>s like<br />

Skype, Facetime or Webex have been video enabled. The built in cameras <strong>on</strong> both tablets and<br />

laptops should be sufficient for video c<strong>on</strong>ference needs using tools provided by the vendors.<br />

49


6. C<strong>on</strong>nectivity<br />

Mitch Cochran and Paul Witman<br />

Tablets can come with WiFi and cellular ph<strong>on</strong>e capabilities. There are no Ethernet c<strong>on</strong>necti<strong>on</strong>s. The<br />

performance will then be limited to the speed and latency WiFi or 4G technologies. Currently there is<br />

no method to c<strong>on</strong>nect a tablet to an Active Directory based organizati<strong>on</strong>.<br />

The issue for organizati<strong>on</strong>s is to develop a plan for c<strong>on</strong>nectivity that meets their needs. WiFi can<br />

present a problem in rolling out WiFi to designated areas. Cellular ph<strong>on</strong>e technology presents the<br />

issue of choosing 3G or 4G vendors based <strong>on</strong> availability and cost.<br />

Once the tablet is c<strong>on</strong>nected to the network, it needs to c<strong>on</strong>nect to either an outside email or a<br />

gateway to reach internal applicati<strong>on</strong>s. If the unit is going to be used as a virtual terminal the gateway<br />

needs to provide that functi<strong>on</strong>, such as a Citrix client or a web-based client. Juniper provides access<br />

to their SSL VPN device for file access and browsing but terminal services is not supported <strong>on</strong><br />

android. The organizati<strong>on</strong> will need to pick a virtual client applicati<strong>on</strong> that will work with the tablet<br />

operating system.<br />

7. Security<br />

From a security perspective, it can be argued that tablets are more secure than laptops. Sensitive<br />

data would be accessed and left <strong>on</strong> the organizati<strong>on</strong>’s gateway. It would not be downloaded or<br />

maintained in the local device.<br />

Too many people believe that Apple based products d<strong>on</strong>’t get viruses. Kevin Haley, Symantec<br />

Director for Security Resp<strong>on</strong>se pointed out that hackers write viruses where they are easy to<br />

implement [3]. Virus’s for Apple products are already in the wild, particularly for the Mac. Currently,<br />

applicati<strong>on</strong>s that can be purchased for Android are available for free <strong>on</strong> various web servers. Many of<br />

those free applicati<strong>on</strong>s include malware comp<strong>on</strong>ents. Kevin also notes that virus writers are attacking<br />

small and medium size businesses. In his blog he lists some protective steps: [4]<br />

Only use app marketplace hosted by well-known legitimate vendors for download and installing<br />

apps.<br />

If practical, adjust Android OS applicati<strong>on</strong> setting to stop the installati<strong>on</strong> of n<strong>on</strong>-market apps.<br />

Review other users’ comment <strong>on</strong> the marketplace to assist in determining if an app is safe before<br />

downloading<br />

During the installati<strong>on</strong> of apps, always check the access permissi<strong>on</strong> being requested for<br />

installati<strong>on</strong>; if they seem excessive for what the applicati<strong>on</strong> is designed to do, it would be wise to<br />

not install the applicati<strong>on</strong>s<br />

Utilize a mobile security soluti<strong>on</strong> to devices to ensure any downloaded apps are not malicious<br />

C<strong>on</strong>sider implement a mobile management soluti<strong>on</strong> to ensure all devices that c<strong>on</strong>nect to their<br />

networks are policy compliant and free of malware.<br />

The industry needs to define how anti-virus and anti-malware software should be implemented. Some<br />

vendors do have anti-virus software available for tablets. Vendors such as Air Watch, Good<br />

Technologies, Mobile Ir<strong>on</strong>, and Sybase provide applicati<strong>on</strong>s for mobile device management and<br />

protecting against data leakage. Data leakage is the accidental or deliberate transfer of corporate<br />

data from the device.<br />

The industry needs to c<strong>on</strong>sider if the tablet is managed like a ph<strong>on</strong>e or a laptop. Each format has<br />

different security or management capabilities. Cell ph<strong>on</strong>es can be managed remotely. The<br />

organizati<strong>on</strong>s need to be able to remotely turn off or wipe clean a tablet in the same way it manages a<br />

cell ph<strong>on</strong>e.<br />

The organizati<strong>on</strong> needs to decide how it will handle users loading pers<strong>on</strong>al applicati<strong>on</strong>s. Most<br />

businesses have policies that prevent the installati<strong>on</strong> of n<strong>on</strong> business approved applicati<strong>on</strong>s. An issue<br />

arises when the device is provided by the user, not the company. Does the organizati<strong>on</strong> have the<br />

capability to prevent a user from loading software <strong>on</strong> a device that the user owns? Many<br />

organizati<strong>on</strong>s are working <strong>on</strong> a BYOT or ‘Bring Your Own Technology’ policy to address what<br />

applicati<strong>on</strong>s the user can load <strong>on</strong> a device that is not provided by the organizati<strong>on</strong>. Should the<br />

organizati<strong>on</strong> choose to, it should be able to enable a policy that prevents the downloading of<br />

50


Mitch Cochran and Paul Witman<br />

applicati<strong>on</strong>s. The organizati<strong>on</strong> should add statements to explicitly state that apps can or cannot be<br />

purchased and installed. Business data and apps can be separated from pers<strong>on</strong>al applicati<strong>on</strong>s [1].<br />

VMWare has announced the Mobile Virtualizati<strong>on</strong> platform for Android devices which lets users run<br />

native applicati<strong>on</strong>s within a secured c<strong>on</strong>tainer. The policy should also address issues such as ediscovery<br />

of organizati<strong>on</strong> documents and the ability to remotely disable and or erase the device.<br />

Cameras could also present a security issue. Some organizati<strong>on</strong>s, such as the FBI, have procedures<br />

that d<strong>on</strong>’t allow cameras in various areas. At <strong>on</strong>e point, they would require the user to surrender any<br />

cell ph<strong>on</strong>e that had an internal camera. It would be returned at the end of the visit. Tablets could<br />

present a similar security issue for organizati<strong>on</strong>s c<strong>on</strong>cerned about cameras.<br />

8. Applicati<strong>on</strong>s<br />

Applicati<strong>on</strong>s <strong>on</strong> a tablet are loaded and tend to be easier to use than those typically loaded <strong>on</strong> a<br />

laptop. Laptop users need to install, learn and then use software. There are more capabilities so<br />

users need to be trained to be aware of and use all of the functi<strong>on</strong>ality. The initial app development for<br />

tablets has been built largely out of the c<strong>on</strong>sumer arena. The applicati<strong>on</strong>s are focused <strong>on</strong> pers<strong>on</strong>al<br />

informati<strong>on</strong>. There are very few applicati<strong>on</strong>s that are focused <strong>on</strong> the organizati<strong>on</strong> or provide for<br />

collaborati<strong>on</strong>.<br />

Many of the applicati<strong>on</strong>s are simple, specialized web sites. These applicati<strong>on</strong>s require access to the<br />

Internet. Some applicati<strong>on</strong>s are simplified versi<strong>on</strong>s of laptop software. Photoshop Elements is<br />

available for tablets but not the full PhotoShop applicati<strong>on</strong>. Microsoft Office provides a limited functi<strong>on</strong><br />

versi<strong>on</strong> for tablets. The standard office product seems to be ‘Documents to Go’ which provides some<br />

limited document capabilities. Many devices provide this office applicati<strong>on</strong> in their standard offering.<br />

One benefit of both smart ph<strong>on</strong>es and tablets is to provide domain specific informati<strong>on</strong> similar to what<br />

can be found <strong>on</strong> a specialized encyclopedia. The City of M<strong>on</strong>rovia Fire Department in California has<br />

standardized <strong>on</strong> a number of applicati<strong>on</strong>s that provide specialized informati<strong>on</strong> that a Divisi<strong>on</strong> Chief<br />

can use <strong>on</strong> an accident scene such as: Los Angeles County fire stati<strong>on</strong> informati<strong>on</strong>, heli-spot<br />

informati<strong>on</strong>, incident reporting cheat sheet, GPS locati<strong>on</strong>, and modern vehicle informati<strong>on</strong> to<br />

determine how to safely avoid airbags or batteries when cutting open a vehicle.<br />

Generally, games are not c<strong>on</strong>sidered to be standard software for organizati<strong>on</strong>s. The tablets are<br />

limited in the graphic capabilities which limits what games can be played. C<strong>on</strong>sider that training<br />

software has taken many of the cues from the sophisticati<strong>on</strong> of gaming software so it will be limited <strong>on</strong><br />

tablet use.<br />

Organizati<strong>on</strong>s are now learning the benefits of collaborative software such as SharePoint. The tablets<br />

have not yet been able to take advantage as full partners in a SharePoint envir<strong>on</strong>ment.<br />

Instant messaging is also an issue. Some instant messaging services are not available across all<br />

platforms. Should the organizati<strong>on</strong> use instant messaging, they would need to choose the platform<br />

that will work with their workstati<strong>on</strong> inventory.<br />

One of the marketing points against an iPad is Apple’s decisi<strong>on</strong> to not support Flash-based web sites.<br />

A c<strong>on</strong>sumer might view the limitati<strong>on</strong> as a weakness since they want to be able to see all web sites.<br />

From the user’s perspective, it would be nice to have as flexible a device as possible. From an<br />

organizati<strong>on</strong>’s point of view, if flash is not used <strong>on</strong> the expected web sites, then exclusi<strong>on</strong> of Flashbased<br />

web sites is not a limitati<strong>on</strong>.<br />

Some tablets have limitati<strong>on</strong>s <strong>on</strong> where software can be obtained. For general governance, this<br />

aspect is a positive for the organizati<strong>on</strong>. The applicati<strong>on</strong>s are c<strong>on</strong>sistent and would generally be ‘safe’<br />

to load <strong>on</strong> a device since the app store has become a trusted source. From a competiti<strong>on</strong> point of<br />

view, it limits the availability of applicati<strong>on</strong>s. The organizati<strong>on</strong> needs to c<strong>on</strong>trol where applicati<strong>on</strong>s are<br />

available from. Marketers are suggesting that organizati<strong>on</strong>s create their own app stores.<br />

C<strong>on</strong>sider that the manufacturers are in the business to make m<strong>on</strong>ey selling hardware. They are not<br />

interested in providing software for all platforms. A quote from Steve Jobs about Apple’s philosophy is<br />

interesting: [2]<br />

51


Mitch Cochran and Paul Witman<br />

"We thought about whether we should do a music client for Android. We put iTunes <strong>on</strong><br />

Windows in order to sell more iPods. But I d<strong>on</strong>'t see an advantage of putting our own<br />

music app <strong>on</strong> Android, except to make Android users happy. And I d<strong>on</strong>'t want to make<br />

Android users happy."<br />

Some organizati<strong>on</strong>s have emails sent from a cell ph<strong>on</strong>e to have a tag line that states ‘sent from my<br />

blackberry’. The iPads can have a ‘sent from my iPad’. The organizati<strong>on</strong> needs to develop a standard<br />

for email formats that is c<strong>on</strong>sistent or uniform across the various devices if they choose to have the<br />

tag line at all.<br />

9. So which device fits where?<br />

We would propose that the smart ph<strong>on</strong>es, tablets and laptops can all have a place in the organizati<strong>on</strong><br />

based <strong>on</strong> the particular needs of each user. Table 3 lists what user functi<strong>on</strong>s are available <strong>on</strong> smart<br />

ph<strong>on</strong>es, tablets, and laptops.<br />

Table 3: User functi<strong>on</strong>s by device<br />

Functi<strong>on</strong> Smart Ph<strong>on</strong>e Tablet Laptop<br />

Check e-mail X X X<br />

Simple Applicati<strong>on</strong>s X X X<br />

View Documentati<strong>on</strong> X X<br />

Use as a gateway to access organizati<strong>on</strong> applicati<strong>on</strong>s X X<br />

Light c<strong>on</strong>tent creati<strong>on</strong> X X<br />

Business web applicati<strong>on</strong> Clients X X<br />

Presentati<strong>on</strong> tool X X<br />

Graphic Design X<br />

Software - ‘Fat’ or ‘Thick’ Clients X<br />

Heavy C<strong>on</strong>tent Creati<strong>on</strong> X<br />

Require CD / DVD X<br />

Require mass storage X<br />

A smart ph<strong>on</strong>e shines for very simple informati<strong>on</strong> lookup or entry <strong>on</strong> a very small screen. The smart<br />

ph<strong>on</strong>es have been very successful for years editing email. Almost all smart ph<strong>on</strong>es include a<br />

reas<strong>on</strong>able camera for quick photos. New apps have appeared that are specialized for the ph<strong>on</strong>es<br />

such as a citizen complaint applicati<strong>on</strong> where the citizen takes a photo of an issue and it is sent<br />

directly to the correct organizati<strong>on</strong>. The key limitati<strong>on</strong>s are processing power, small screen and<br />

multitasking.<br />

Tablets can be thought of as large cell ph<strong>on</strong>es. They use many of the same apps but provide<br />

processing power, expanded storage, a larger screen and keyboard.<br />

Tablets have been shown to have a positive impact <strong>on</strong> student interacti<strong>on</strong>. Wolf found that using<br />

tablets in the classroom yielded more timely interesting lectures which resulted in increased student<br />

performance [5]. Many cities have implemented tablets as the delivery vehicle for council documents.<br />

The council members are able to view and mark up PDF based documents with their notes prior to<br />

council meetings. Building <strong>on</strong> the findings from Wolf and others [5] [6], and the city’s council<br />

applicati<strong>on</strong> experience, it makes sense that tablets could serve as electr<strong>on</strong>ic books or training tools.<br />

10. Future<br />

The tablet should follow the same growth path as computers have. It is safe to assume that the<br />

processors will increase in power, the memory will increase and the storage capacity will increase.<br />

The amount of sales will have an impact as to how fast the units evolve. The competiti<strong>on</strong> between the<br />

two operating systems should drive the competitors into a game of <strong>on</strong>e-ups-man-ship that we have<br />

seen in other historical developments such as the browser wars between Netscape and Microsoft.<br />

It is expected that software will start to be available across all platforms. Microsoft has announced<br />

Windows 8 which will include cross platform capabilities. They have just introduced the ‘Surface’ as<br />

their entry into the tablet arena. They have models that are focused at the very low end devices such<br />

as the kindle and models that are targeted against laptops. The advantage for enterprise is that<br />

corporate applicati<strong>on</strong>s that require a Windows based thick client can be directly supported instead of<br />

using a web based fr<strong>on</strong>t end.<br />

52


Mitch Cochran and Paul Witman<br />

Since many applicati<strong>on</strong>s are web based, the <strong>on</strong>ly changes that may be needed are changing the<br />

screen design for smaller or larger screens. This situati<strong>on</strong> has been dem<strong>on</strong>strated as websites can<br />

detect the viewing device and can automatically direct the user to the correct web site. The vendor,<br />

Bluestack has developed software that will allow Android applicati<strong>on</strong>s to run <strong>on</strong> windows.<br />

Voice recogniti<strong>on</strong> will also move across the computing platforms. The Apple iPh<strong>on</strong>e 4S sets the<br />

standards or customer expectati<strong>on</strong>s with its implementati<strong>on</strong> of a intelligent voice applicati<strong>on</strong>, Siri. It is<br />

safe to assume that the voice recogniti<strong>on</strong> will move across all of the platforms as the technology<br />

matures.<br />

11. C<strong>on</strong>clusi<strong>on</strong><br />

Tablets definitely have a place in the organizati<strong>on</strong>’s computer inventory. The questi<strong>on</strong> for<br />

differentiati<strong>on</strong> is to find where is the ‘sweet spot’ or most appropriate locati<strong>on</strong> for this technology? One<br />

of the <strong>on</strong>going themes is that tablets are more appropriate for c<strong>on</strong>tent lookup where the laptop is<br />

better positi<strong>on</strong>ed for c<strong>on</strong>tent creati<strong>on</strong>. Over time, tablets will be released that have more processing<br />

power and memory. Those increased attributes will allow for tablet manufacturers to standardize both<br />

voice and handwriting recogniti<strong>on</strong> opti<strong>on</strong>s.<br />

Software or functi<strong>on</strong>al developers need the software tools, memory and storage that laptops can<br />

provide. Of course a tablet could be used for quick development but laptops are better suited for<br />

l<strong>on</strong>ger term activities. The developers also need the larger screen size.<br />

From a governance perspective, the organizati<strong>on</strong> needs to c<strong>on</strong>sider some of the same issues in<br />

governing cell ph<strong>on</strong>es:<br />

Limit the installati<strong>on</strong> of applicati<strong>on</strong>s by the user<br />

Capability to provide a c<strong>on</strong>sistent software image<br />

Have the ability to track the devices<br />

Have the ability to remotely wipe off or delete the software from the device<br />

C<strong>on</strong>trol what networks the device can attach to<br />

At some point the two different distincti<strong>on</strong>s will merge into <strong>on</strong>e design with the <strong>on</strong>ly separati<strong>on</strong> being<br />

an attached keyboard. Microsoft has announced its goal of having Windows 8 being capable of<br />

working across multiple platforms. The new operating system is planned to provide an app store and<br />

tablet support.<br />

12. Future research directi<strong>on</strong>s<br />

This paper introduces the topic and the initial governance issues. The acceptance of tablets in the<br />

enterprise signifies a change in the traditi<strong>on</strong>al computing governance model. There are numerous<br />

areas for further research <strong>on</strong> how tablets are being used which include:<br />

Case studies of organizati<strong>on</strong>s integrating pers<strong>on</strong>al devices<br />

Survey of governance models used<br />

Survey of technologies used to implement governance models<br />

Integrati<strong>on</strong> of task-technology fit theories<br />

Re-assess as device capabilities improve and evaluate add-<strong>on</strong> comp<strong>on</strong>ents such as keyboards<br />

become more ubiquitous.<br />

References<br />

Anders<strong>on</strong>, R., Anders<strong>on</strong>, R., Linnell, N., Razmov, V., (2006) Supporting structured activities and collaborati<strong>on</strong><br />

through the use of student devices in college classrooms, Manuscript,<br />

http://www.cs.washingt<strong>on</strong>.edu/educati<strong>on</strong>/dl/presenter/papers/2006/AALR_2006.pdf,<br />

Ant Ozok, A., Bens<strong>on</strong>, Dana, Chakraborty, Joyram, Norcio, Anth<strong>on</strong>y F., (2011) A comparative Study Between<br />

Tablet and Laptop PCs: User Satisfacti<strong>on</strong> and Preferences, Internati<strong>on</strong>al Journal of Human-Computer<br />

Interacti<strong>on</strong>, 24:3, 329-352. http://dx.doi.org/10.1080/10447310801920524.<br />

Currier, Guy, Emerging Technology Adopti<strong>on</strong> Trends, CIO Insight, September/October 2011, pp 18-23.<br />

Blacharski, D, (2011) Using the tablet as an enterprise business tool ,<br />

http://www.<strong>on</strong>estopclick.com/blog/index.php/2011/11/using-the-tablet-as-an-enterprise-business-tool/,<br />

November 1.<br />

53


Mitch Cochran and Paul Witman<br />

Currier, Guy, (2011) The Four Fastest-Growing Technology Areas,, Baseline Magazine,.<br />

http://betweenthelines.baselinemag.com/c<strong>on</strong>tent/trends/the_four_fastest-growing_technology_areas.html.<br />

November 4.<br />

Dimensi<strong>on</strong>al Research, (2011) Enterprise ipad and Tablet Adopti<strong>on</strong>: A Survey, Forrester, tablet usage am<strong>on</strong>g<br />

<strong>on</strong>line buyers. http://www.slideshare.net/BizrateInsights/bizrateforrester-study-tablet-usage-am<strong>on</strong>g-<strong>on</strong>linebuyers.<br />

www.modelmetrics.com/wp-c<strong>on</strong>tent/.../05/iPadSurvey-May10.pdf, May.<br />

Goodhue, D., (2011) The Model Underlying the Measurement of the Impacts of the IIC <strong>on</strong> the End-Users, journal<br />

of the American society of rInfomrati<strong>on</strong> science, 48(5): 449-453, 1997.<br />

Haley, Kevin, (2011) IEEE Security Presentati<strong>on</strong>, Clarem<strong>on</strong>t Graduate University, August.<br />

Haley, Kevin, (2011) Symantec Security Resp<strong>on</strong>se Director, Symantec Blog,<br />

www.symantec.com/c<strong>on</strong>nect/blogs/mobile-malware-do-your-employees-know-what-look, October.<br />

Horwitt, Elisabeth, (2011) Mobility, CIO Magazine, October 15, 2011, p. 32.<br />

Isaacs<strong>on</strong> Walter, (2012) Steve Jobs: A Biography, Sim<strong>on</strong> & Schuster, 2012<br />

Willis, D., (2011) iPad and Bey<strong>on</strong>d: The Media Tablet in Business, Gartner,<br />

http://www.gartner.com/it/c<strong>on</strong>tent/1586600/1586614/april_13_ipad_and_bey<strong>on</strong>d_dwillis.pdf,.<br />

Wolf, T., (2007) Assessing the impact of inking technology in a large digital design course. Proceedings of the<br />

38th SIGCSE Technical Symposium <strong>on</strong> Computer Science Educati<strong>on</strong>, 39(1), 79-83.<br />

54


Classifying IT Investment Evaluati<strong>on</strong> Methods According to<br />

Functi<strong>on</strong>al Criteri<strong>on</strong><br />

Jacek Cypryjanski<br />

University of Szczecin, Poland<br />

jacek.cypryjanski@wneiz.pl<br />

Abstract: One of the main problems of IT investment evaluati<strong>on</strong> is selecti<strong>on</strong> of adequate methods. The selecti<strong>on</strong><br />

will be easier if we divide the methods into homogeneous groups. The paper presents a proposal of methods<br />

classificati<strong>on</strong> based <strong>on</strong> functi<strong>on</strong>ality criteri<strong>on</strong>. As a starting point the system-situati<strong>on</strong>al approach has been<br />

adopted. Development of this approach resulted in three findings which form a theoretical basis of the proposed<br />

classificati<strong>on</strong>: (a) model of the evaluati<strong>on</strong> system with three phases of the evaluati<strong>on</strong> process and relati<strong>on</strong>ships<br />

between them; (b) synthetic approach to problems of evaluati<strong>on</strong> in general and identificati<strong>on</strong> of problem sources<br />

typical for IT investments; (c) synthetic approach to requirements which the appraisal (result of the evaluati<strong>on</strong>)<br />

should fulfil in a decisi<strong>on</strong> process. The proposed classificati<strong>on</strong> organizes the problem of evaluati<strong>on</strong> method<br />

selecti<strong>on</strong>. It allows avoiding series of mistakes such as perceiving methods which are of different roles as<br />

alternative <strong>on</strong>es or underrating methods which could be of great importance in evaluati<strong>on</strong> of IT investments.<br />

Keywords: IT investment evaluati<strong>on</strong>, evaluati<strong>on</strong> methods classificati<strong>on</strong>, specificity of IT investments<br />

1. Introducti<strong>on</strong><br />

A big part of the evaluati<strong>on</strong> of IT investments literature is dedicated to methods. Berghout and<br />

Renkema (2001) list over sixty methods, that all to be of help in the evaluati<strong>on</strong> process. There are<br />

proposals of new methods which in certain c<strong>on</strong>diti<strong>on</strong>s bring positive results, e.g. Informati<strong>on</strong><br />

Ec<strong>on</strong>omics (Parker et al., 1988, 1989). There are also examples of how positive results were<br />

achieved using methods adopted from other fields, e.g. financial engineering (Dos Santos 1991,<br />

Kambil et al. 1993, Kumar 1996, Lee and Lee 2011) or even classic methods of investments<br />

evaluati<strong>on</strong> (Farbey et al. 1992, Botchkarev and Andru 2011). These achievements cannot be<br />

overestimated; however, they paradoxically complicate the search for evaluati<strong>on</strong> method adequate to<br />

a given case. Especially if we emphasise the fact that all these findings are dedicated to practiti<strong>on</strong>ers<br />

who do not have as much time to study literature as researches. It is like pumping up new ideas into<br />

an unorganized domain – it heightens chaos. However, am<strong>on</strong>g the new methods there are excepti<strong>on</strong>s<br />

which put things in order. These are, using terms from tax<strong>on</strong>omy proposed by Bannister and Remenyi<br />

(2000), meta approaches as the <strong>on</strong>es by Farbey et al. (1993, 1999), which play different roles in the<br />

evaluati<strong>on</strong> process: attempt to select the optimum set of measures for a c<strong>on</strong>text or set of<br />

circumstances. It’s worthwhile putting a questi<strong>on</strong> whether other functi<strong>on</strong>s of evaluati<strong>on</strong> process could<br />

be distinguished and can other methods be attributed to them?<br />

The answer seems to be obvious if we compare for example cost-benefits analysis (CBA) and return<br />

<strong>on</strong> investment (ROI) – two methods which are often menti<strong>on</strong>ed in the c<strong>on</strong>text of IT investment<br />

evaluati<strong>on</strong>. According to Ferbey et al. (1999), “CBA is an approach that attempts to find a m<strong>on</strong>ey<br />

value for each element c<strong>on</strong>tributing to the cost and benefit of a development project. The approach<br />

originated as an attempt to deal with the problem that some elements regarded as benefits or costs<br />

have no obvious market value or price. (...) The resulting cost-benefit values can be projected in the<br />

form of noti<strong>on</strong>al cash flows <strong>on</strong> a year-by year basis and the projected outcomes for alternative<br />

schemes or designs fed into a decisi<strong>on</strong> model based <strong>on</strong> <strong>on</strong>e of the standard ROI methods.” In other<br />

words CBA and ROI play different roles in evaluati<strong>on</strong> process (the first helps us to deal with<br />

immeasurability, while the sec<strong>on</strong>d is a method for calculating efficiency of an investment), they are<br />

complementary to each other, and therefore shouldn’t be treated as an alternative.<br />

The aim of the study presented in the paper is to develop a new classificati<strong>on</strong> of methods based <strong>on</strong><br />

functi<strong>on</strong>ality criteri<strong>on</strong>. In order to classify methods according to the functi<strong>on</strong> they play in an evaluati<strong>on</strong><br />

process, the system and the situati<strong>on</strong>al approaches have been adopted. Using system approach<br />

allows the better to understand the significance of an envir<strong>on</strong>ment of the examined subject and the<br />

relati<strong>on</strong>ships between them. It also allows defining elements of the examined subject, to perceive<br />

them as individual systems and as subsystems of a broader system. It helps to define the relati<strong>on</strong>s<br />

between the subsystems – how they affect <strong>on</strong>e another and what is the synergy effect of their<br />

interacti<strong>on</strong>.<br />

55


Jacek Cypryjanski<br />

Situati<strong>on</strong>al approach is as it were the supplement of the system approach. It was formulated in the<br />

1970s within the management theory. According to Kast and Rosenzweig (1973) the situati<strong>on</strong>al<br />

approach to organizati<strong>on</strong>s and their management assumes that an organizati<strong>on</strong> is a system<br />

c<strong>on</strong>sisting of subsystems and is separated from the envir<strong>on</strong>ment with clearly identifiable boundaries.<br />

The situati<strong>on</strong>al point of view tries to understand interrelati<strong>on</strong>s between and inside the subsystems as<br />

well as between the organizati<strong>on</strong>s and their envir<strong>on</strong>ment. It tries to define the relati<strong>on</strong>ships model and<br />

variables’ c<strong>on</strong>figurati<strong>on</strong>s. It emphasises a multivariate nature of organizati<strong>on</strong> structure and tries to<br />

understand how organizati<strong>on</strong>s operate in different c<strong>on</strong>diti<strong>on</strong>s and various circumstances. Situati<strong>on</strong>al<br />

approach heads for explorati<strong>on</strong> of organizati<strong>on</strong>al structure and management techniques tailored to the<br />

specific situati<strong>on</strong>.<br />

Situati<strong>on</strong>al approach has been successfully adopted to solve problems of IT investments evaluati<strong>on</strong><br />

(Farbey et al. 1993, 1999, Peters 1994, 1996). As noted Serafeimidis (2001): „There is a clear need<br />

for c<strong>on</strong>tingent evaluati<strong>on</strong> approaches in order to deal with the range of circumstances encountered.<br />

This implies that IT projects, as well as their c<strong>on</strong>texts have certain characteristics which influence the<br />

choice of a suitable evaluati<strong>on</strong> method. Similarly, every evaluati<strong>on</strong> methodology or technique has<br />

characteristics which point to the set of circumstances in which it could be applied more successfully.”<br />

Development of this approaches resulted in three findings which form a theoretical basis of the<br />

proposed classificati<strong>on</strong>:<br />

model of the evaluati<strong>on</strong> system with three phases of the evaluati<strong>on</strong> process and relati<strong>on</strong>ships<br />

between them;<br />

synthetic approach to problems of evaluati<strong>on</strong> in general and identificati<strong>on</strong> of problem sources<br />

typical for IT investments;<br />

Synthetic approach to requirements which the appraisal (result of the evaluati<strong>on</strong>) should fulfil in a<br />

decisi<strong>on</strong> process.<br />

The next three secti<strong>on</strong>s describe the above menti<strong>on</strong>ed findings. The fifth secti<strong>on</strong> presents the<br />

proposed classificati<strong>on</strong> and describes its relati<strong>on</strong>s with tax<strong>on</strong>omy of techniques drawn up by Bannister<br />

and Remenyi.<br />

2. Evaluati<strong>on</strong> as a system<br />

To present an evaluati<strong>on</strong> as a system <strong>on</strong>e should define its goal, the subsystems it c<strong>on</strong>sists of and<br />

interrelati<strong>on</strong>s between them. The system’s envir<strong>on</strong>ment should be defined and how it interacts with<br />

the system. It is important to notice that the definiti<strong>on</strong> of system does not follow any strict rules or<br />

principles but should be guided by the stated purpose of the study (Katz and Kahn, 1980, Jokela et al.<br />

2008). Here the evaluati<strong>on</strong> is seen as a system which goal is to generate the appraisal that meets two<br />

c<strong>on</strong>diti<strong>on</strong>s: unambiguously and objectively reflects the actual state and is significant in the decisi<strong>on</strong><br />

process. The evaluati<strong>on</strong> system c<strong>on</strong>sists of four subsystems: input, subsystem performing the<br />

evaluati<strong>on</strong> process, c<strong>on</strong>trol subsystem and output (figure 1).<br />

Subsystem performing the evaluati<strong>on</strong> process, in turn, c<strong>on</strong>sists of three subsystems carrying out the<br />

following tasks:<br />

identificati<strong>on</strong> of all relevant factors and relati<strong>on</strong>ships between them,<br />

quantificati<strong>on</strong> of factors and relati<strong>on</strong>ships in such measurement scale that is proper for generating<br />

appraisal,<br />

Generati<strong>on</strong> of the appraisal.<br />

In the model shown in figure 1 these subsystems (evaluati<strong>on</strong> phases, functi<strong>on</strong>s of evaluati<strong>on</strong> process)<br />

were named respectively: understanding, measurement, and assessment, as in the definiti<strong>on</strong> by<br />

Remenyi et al. (1997): “evaluati<strong>on</strong> is a series of activities incorporating understanding, measurement,<br />

and assessment. It is either a c<strong>on</strong>scious or tacit process which aims to establish the value of or the<br />

c<strong>on</strong>tributi<strong>on</strong> made by a particular situati<strong>on</strong>. It can also relate to the determinati<strong>on</strong> of the worth of an<br />

object.”<br />

Decisi<strong>on</strong> process, for which the evaluati<strong>on</strong> is carried out, determines the way how assessment<br />

subsystem generates the appraisal. For example while making an investment decisi<strong>on</strong>, <strong>on</strong>e should<br />

evaluate the investment efficiency in order to know whether the benefits are worth the investment<br />

expenditures? In this case assessment may c<strong>on</strong>sist in the calculati<strong>on</strong> of return of investment. In order<br />

56


Jacek Cypryjanski<br />

to reflect reality objectively, ROI has to include all comp<strong>on</strong>ents of costs and benefits. It means that<br />

understanding subsystem requires acti<strong>on</strong>s allowing identificati<strong>on</strong> of all costs and benefits while<br />

measurement subsystem expressing them in m<strong>on</strong>etary terms. If measurement subsystem cannot<br />

express some benefits (e.g. higher customer satisfacti<strong>on</strong>, more accurate decisi<strong>on</strong>s) in m<strong>on</strong>ey, two<br />

scenarios are possible. The first is that understanding subsystem will search for measurable variables<br />

to describe given benefit. In this case the quality of the generated appraisal will depend <strong>on</strong> how well<br />

the selected variables describe the benefit. In the sec<strong>on</strong>d scenario we do not calculate ROI in the<br />

assessment subsystem. Instead, total investment costs are calculated while benefits are presented in<br />

a descriptive form. Of course it is not a formal efficiency appraisal yet, however it gives decisi<strong>on</strong><br />

maker an idea about investment efficiency.<br />

Figure 1: Evaluati<strong>on</strong> as a system<br />

This simple example illustrates two issues. First of all it describes relati<strong>on</strong>s between understanding,<br />

measurement and assessment subsystems. It shows how acti<strong>on</strong>s taken and methods used in <strong>on</strong>e<br />

subsystem (phase of evaluati<strong>on</strong> process) affect the way of performing and methods used in other<br />

subsystems. It also clarifies that decisi<strong>on</strong>s about acti<strong>on</strong>s carried out in these subsystems are taken by<br />

a compromise between the objective reflecti<strong>on</strong> of reality (search for measurable variables to describe<br />

given benefit) and usefulness in decisi<strong>on</strong> process (instead of ROI, calculating total investment costs<br />

and presenting benefits in a descriptive form). Sec<strong>on</strong>dly, it shows envir<strong>on</strong>mental influence <strong>on</strong> the<br />

evaluati<strong>on</strong> system. On the <strong>on</strong>e hand a decisi<strong>on</strong> process determines the requirements which an<br />

appraisal should fulfil to be useful. On the other hand what we evaluate and when we do it causes<br />

problems with obtaining such an appraisal. These problems - specific to individual cases - are named<br />

here as c<strong>on</strong>diti<strong>on</strong>s of evaluati<strong>on</strong>. Selecti<strong>on</strong> of methods for evaluati<strong>on</strong> process tasks as well as criteria<br />

and measures should be d<strong>on</strong>e through analysis of the requirements and c<strong>on</strong>diti<strong>on</strong>s. Hence the c<strong>on</strong>trol<br />

subsystem presented in the model (Figure 1) is shown as a system carrying out four tasks:<br />

requirements and c<strong>on</strong>diti<strong>on</strong>s analysis,<br />

criteria/measurement selecti<strong>on</strong>,<br />

defining evaluati<strong>on</strong> activities,<br />

Selecting methods for activities realizati<strong>on</strong>.<br />

These tasks, analogically to the tasks of the subsystem performing the evaluati<strong>on</strong> process should be<br />

seen as subsystems which interact <strong>on</strong> each other. Later in this paper we look closely at c<strong>on</strong>diti<strong>on</strong>s<br />

and requirements as important arguments in favor of the proposed classificati<strong>on</strong>.<br />

57


3. C<strong>on</strong>diti<strong>on</strong>s<br />

Jacek Cypryjanski<br />

Problems of evaluati<strong>on</strong> can be assigned to <strong>on</strong>e of the three categories:<br />

indeterminacy – problems with identificati<strong>on</strong> of all factors relevant to evaluati<strong>on</strong> and relati<strong>on</strong>ships<br />

between them,<br />

variability – problems of determining how all factors and relati<strong>on</strong>ships between them will change<br />

over time,<br />

Immeasurability – problems with quantificati<strong>on</strong> of all factors and relati<strong>on</strong>ships in such<br />

measurement scale that is proper for generating appraisal.<br />

In general indeterminacy and variability are perceived collectively as uncertainty. But when we<br />

analyze the problems of evaluati<strong>on</strong> in the c<strong>on</strong>text of applicable methods then this distincti<strong>on</strong> becomes<br />

important. Different methods are used for dealing with indeterminacy, e.g. system analysis, and<br />

different for variability, e.g. statistical methods. Indeterminacy, variability and immeasurability may<br />

occur with different intensities, and each time dealing with any of them may be crucial for the<br />

evaluati<strong>on</strong>. Once the priority will be analyzing cause and effect relati<strong>on</strong>ships between changes in an<br />

organizati<strong>on</strong>’s IS and effectiveness of business processes (indeterminacy). Another time it will be<br />

searching for quantitative variables (ratios) which best reflect different quality categories in a given<br />

situati<strong>on</strong>, e.g. customer satisfacti<strong>on</strong>, employee satisfacti<strong>on</strong> or corporate knowledge (immeasurability).<br />

Yet another time it will be the problem of identificati<strong>on</strong> of events which result in the fact that effects<br />

and investment expenditure may differ from the expected (variability). The listed problems and their<br />

scale form specific c<strong>on</strong>diti<strong>on</strong>s of a particular evaluati<strong>on</strong>.<br />

Are these categories of problems specific to IT investment evaluati<strong>on</strong>? Powell (1999) remarks that<br />

many sources (Strassman 1997, Willcocks 1996, Wen and Sylla 1999) “suggest that IS/IT investment<br />

is different from other investment decisi<strong>on</strong>s because the costs and benefits are harder to identify and<br />

quantify and the intangible factors are likely to be significant.” However he also notes that “the idea<br />

that the measurement of costs and benefits is problematic has surfaced in fields other then IS/IT”. It<br />

would be difficult to accept that indeterminacy, variability and immeasurability are category of<br />

problems specific to any type of evaluati<strong>on</strong>. The problems appear in different evaluati<strong>on</strong> types<br />

including evaluati<strong>on</strong> of IT investments. But without a doubt the higher intensity of problems the more<br />

difficult investment evaluati<strong>on</strong>. The specificity of IT investments evaluati<strong>on</strong> is that indeterminacy,<br />

variability and immeasurability appear in great force and their source are:<br />

wide range and complexity of IT systems’ influence <strong>on</strong> organizati<strong>on</strong>s,<br />

nature of informati<strong>on</strong>,<br />

Rapid IT development.<br />

The wide range of influence arises due to the fact that informati<strong>on</strong> systems realise informati<strong>on</strong><br />

processes which <strong>on</strong> the other hand are an inseparable item of any activities, both in the field of<br />

management and operati<strong>on</strong>. The complexity of the influence is that informati<strong>on</strong> systems <strong>on</strong>ly enable<br />

achieving benefits which depend <strong>on</strong> many other factors (Lucas 1993, Markus and Soh 1993,<br />

Mahmood and Mann 2000, Remenyi et al. 2000). As Remenyi et al. (2000) pointed out: “the value of<br />

the IT investment depends entirely up<strong>on</strong> the way in which it is able to make the organizati<strong>on</strong> more<br />

efficient and effective”. Informati<strong>on</strong> is a unique good, different from material goods. The value of<br />

informati<strong>on</strong> generated by IS may <strong>on</strong>ly be examined in informati<strong>on</strong>-user-task frame of reference. The<br />

phenomen<strong>on</strong> of informati<strong>on</strong> asymmetry makes the ex ante estimati<strong>on</strong> of informati<strong>on</strong> value impossible<br />

wherever the effect of the activities (in which informati<strong>on</strong> is used) is dependant <strong>on</strong> informati<strong>on</strong> c<strong>on</strong>tent.<br />

Value of informati<strong>on</strong> is a highly complex issue (Bannister and Remenyi 2000), especially when<br />

c<strong>on</strong>sidered in c<strong>on</strong>juncti<strong>on</strong> with the wide range of IT systems’ influence <strong>on</strong> organizati<strong>on</strong>s. All<br />

complemented by development of IT, the fastest growing general purpose technology in history<br />

(Jovanovic and Rousseau 2005).<br />

Looking into the presented problems from an evaluati<strong>on</strong> system perspective we notice that they<br />

c<strong>on</strong>cern understanding and measurement subsystems. This has also been proved by empirical<br />

studies c<strong>on</strong>ducted by Ballantine et al. (1999). Of course the problems influence the assessment<br />

subsystem as well, but <strong>on</strong>ly indirectly through relati<strong>on</strong>ships between these three subsystems. This<br />

means that the quality of appraisal is largely dependent <strong>on</strong> how understanding and measurement are<br />

realized and which methods of identificati<strong>on</strong> and quantificati<strong>on</strong> have been applied. Therefore while<br />

developing classificati<strong>on</strong>s, methods of identificati<strong>on</strong> and quantificati<strong>on</strong> should be taken into account<br />

58


Jacek Cypryjanski<br />

equally with the methods of assessment. They should also be assigned to individual roles they play in<br />

the process of evaluati<strong>on</strong>.<br />

4. Requirements<br />

The effects of investments as well as any other activities’ may be c<strong>on</strong>sidered in two main aspects: for<br />

the intended purpose and for the efforts. In the first case effectiveness is analysed, i.e. a degree of<br />

goal achievement while the aspect of expenditures is omitted. Only those effects have been taken into<br />

c<strong>on</strong>siderati<strong>on</strong> which were intended. In the sec<strong>on</strong>d case efficiency is analysed, i.e. effects of activities<br />

and their expenditures. All obtained effects are taken into c<strong>on</strong>siderati<strong>on</strong>, no matter whether they were<br />

intended or not. This means that while evaluating effectiveness and efficiency, different scopes of<br />

effects are taken into account. The evaluati<strong>on</strong> of investments requires examining both aspects.<br />

Focusing <strong>on</strong> the first aspect <strong>on</strong>ly may lead to a situati<strong>on</strong> in which expenditures overcame the effects.<br />

Analysing <strong>on</strong>ly the relati<strong>on</strong> between effects and expenditures may cause a situati<strong>on</strong> in which the goal<br />

would not be achieved.<br />

Because of the time of evaluati<strong>on</strong> criteri<strong>on</strong> we distinguish retrospective evaluati<strong>on</strong> (ex post) and<br />

prospective <strong>on</strong>e (ex ante). In fact both evaluati<strong>on</strong>s are carried out ex post, as it is difficult to imagine<br />

to evaluate something that do not exist. The difference lies in the subject of evaluati<strong>on</strong>. In the<br />

retrospective evaluati<strong>on</strong> we asses activities, and what we call an ex ante evaluati<strong>on</strong> is in fact a<br />

retrospective evaluati<strong>on</strong> of intenti<strong>on</strong>s (plans of these activities).<br />

As far as the subject of evaluati<strong>on</strong> is c<strong>on</strong>cerned we distinguish an synthetic evaluati<strong>on</strong> – when all the<br />

activities are evaluated, and fragmentary evaluati<strong>on</strong> – when <strong>on</strong>ly some aspects are evaluated. The<br />

reas<strong>on</strong> for which we evaluate effectiveness and efficiency is another classificati<strong>on</strong> criteri<strong>on</strong>. According<br />

to this criteri<strong>on</strong> we divide evaluati<strong>on</strong> into absolute and relative. The absolute evaluati<strong>on</strong> of<br />

effectiveness allows us to know whether and to what extend the analysed activities lead us to the<br />

intended goal. The absolute evaluati<strong>on</strong> of efficiency let us know whether the benefits are worth the<br />

investment expenditures. Whereas the relative evaluati<strong>on</strong> is about comparing effectiveness and<br />

efficiency of different activities.<br />

Different decisi<strong>on</strong> situati<strong>on</strong>s require different appraisal. First of all, appraisals may differ as far as their<br />

scope is c<strong>on</strong>cerned: <strong>on</strong>ly efficiency is assessed, or <strong>on</strong>ly effectiveness, or both. In spite of the fact that<br />

appraisal of efficiency and effectiveness are complementary, there may be some situati<strong>on</strong>s where<br />

<strong>on</strong>ly <strong>on</strong>e of them is necessary. In some particular cases, e.g. in relative evaluati<strong>on</strong>, where the aim is<br />

dichotomous and the undertaking do not generate any additi<strong>on</strong>al effects, it is enough to estimate and<br />

compare <strong>on</strong>ly investment expenditures. In additi<strong>on</strong>, depending <strong>on</strong> whether the decisi<strong>on</strong> is to accept a<br />

project (such as implementati<strong>on</strong> of CRM system), or to choose <strong>on</strong>e of the possible variants of the<br />

project (e.g. which of the available systems to choose, in what order to implement IS modules, etc.), it<br />

is more or less important that the appraisal is synthetic, and the measure taken as a criteri<strong>on</strong><br />

expresses the phenomena in m<strong>on</strong>ey terms. Making a decisi<strong>on</strong> in the first case requires such an<br />

efficiency assessment that would allow to present the evaluated issue compared to other activities in<br />

a company. This in practice means the necessity of applying financial measures. In the sec<strong>on</strong>d case<br />

the situati<strong>on</strong> allows to accept more fragmentary assessments and measures different to financial<br />

<strong>on</strong>es. These differences in the scope and the way of assessment may be c<strong>on</strong>sidered as requirements<br />

towards generated appraisal.<br />

5. Proposed classificati<strong>on</strong> of methods<br />

As the model of a system in figure 1 shows we can assign methods used in the process of evaluati<strong>on</strong><br />

to particular subsystems. In this way we obtain a classificati<strong>on</strong> of methods according to the criteri<strong>on</strong> of<br />

functi<strong>on</strong>ality (as each of the subsystems performs a different functi<strong>on</strong>). According to this classificati<strong>on</strong><br />

we distinguish four categories of methods:<br />

methods of identificati<strong>on</strong>,<br />

methods of quantificati<strong>on</strong>,<br />

methods of assessment,<br />

Meta approaches.<br />

In practice, a number of methods are composite approaches, e.g. informati<strong>on</strong> ec<strong>on</strong>omics by Parker,<br />

Bens<strong>on</strong> and Trainor or Gartner total cost of ownership. However, creating a separate category for<br />

these methods would destroy the sense of the classificati<strong>on</strong>. In the case of composite approaches<br />

59


Jacek Cypryjanski<br />

which support the realizati<strong>on</strong> of more than <strong>on</strong>e phase of evaluati<strong>on</strong> process we can assign them to<br />

each of these phases. At the same time we can define how a given method supports the realizati<strong>on</strong> of<br />

a specific phase. It is relatively simple as composite methods are usually in the form of procedures.<br />

Meta approaches’ task is to support subsystems of evaluati<strong>on</strong> process c<strong>on</strong>trol. Therefore they should<br />

be perceived as a set of procedures and analytical frameworks supporting requirements and<br />

c<strong>on</strong>diti<strong>on</strong>s analysis, criteria/measurement selecti<strong>on</strong>, definiti<strong>on</strong> of evaluati<strong>on</strong> activities and finally<br />

selecti<strong>on</strong> of methods for activities realizati<strong>on</strong>. A good example of meta approaches is the matching<br />

process developed by Farbey, Land and Targett (1999). It c<strong>on</strong>sists of a three-stage procedure and a<br />

series of matrices (analytical frameworks) which enable requirements and c<strong>on</strong>diti<strong>on</strong>s analysis as well<br />

as method selecti<strong>on</strong>. In the matching process requirements are analyzed based <strong>on</strong> six criteria (matrix<br />

dimensi<strong>on</strong>s). The same is true in the case of c<strong>on</strong>diti<strong>on</strong>s analysis, however we can assume that three<br />

of the matrix dimensi<strong>on</strong>s relate to the problem of indeterminacy (nature of system: specific<br />

infrastructure; directness of impact: direct, indirect; leadership role: follower, leader), two to the<br />

problem of variability (certainty of impact: certain, uncertain; industry situati<strong>on</strong>: stable, turbulent) and<br />

<strong>on</strong>e to the problem with quantificati<strong>on</strong> (type of benefits: quantifiable, qualitative). The selecti<strong>on</strong> of<br />

methods comes down to pointing out <strong>on</strong>e of the four categories of methods. This divisi<strong>on</strong> does not<br />

take into c<strong>on</strong>siderati<strong>on</strong> the functi<strong>on</strong>ality criteri<strong>on</strong> therefore is quite general.<br />

The proposed classificati<strong>on</strong> was created as complementary to tax<strong>on</strong>omy of techniques drawn up by<br />

Bannister and Remenyi (2000). This c<strong>on</strong>cept is illustrated in Figure 2.<br />

Figure 2: Categories of methods and approaches to investment evaluati<strong>on</strong><br />

Tax<strong>on</strong>omy of Bannister and Remenyi classifies approaches to evaluating IT investments into three<br />

basic groups of techniques (fundamental measures, composite approaches and meta approaches)<br />

which can be used in two different ways (positivist/reducti<strong>on</strong>ist and hermeneutic). According to<br />

Bannister and Remenyi (2000) “fundamental measures are metrics which attempt to parameterize<br />

some characteristic or closely related set of characteristics of the investment down to a single<br />

measure (…) composite techniques combine several fundamental measures to obtain a ‘balanced’<br />

overall picture of value/investment return” while “meta approaches attempt to select the optimum set<br />

of measures for a c<strong>on</strong>text or set of circumstances”. The descripti<strong>on</strong>s of the particular categories show<br />

that the authors focus in their tax<strong>on</strong>omy <strong>on</strong> measures of evaluati<strong>on</strong> (which according to functi<strong>on</strong>ality<br />

criteri<strong>on</strong> were defined here as methods of assessment), omitting identificati<strong>on</strong> and quantificati<strong>on</strong><br />

methods (as far as they are not a part of composite methods, as in the case of informati<strong>on</strong><br />

ec<strong>on</strong>omics). It also shows that authors narrowly define meta approaches as methods supporting<br />

60


Jacek Cypryjanski<br />

selecti<strong>on</strong> of measures while here meta approaches support selecti<strong>on</strong> of methods for the whole<br />

evaluati<strong>on</strong> process. The proposed integrati<strong>on</strong> of the two classificati<strong>on</strong>s is probably against the<br />

tax<strong>on</strong>omy authors’ intenti<strong>on</strong>. However, it seems justified if perceiving evaluati<strong>on</strong> from the system<br />

perspective.<br />

6. Summary and c<strong>on</strong>clusi<strong>on</strong><br />

The evaluati<strong>on</strong> process has its specificity (comm<strong>on</strong> and differentiating features of individual cases)<br />

determined by what, why and when is being evaluated. The paper shows that:<br />

The specificity can be expressed as requirements which an evaluati<strong>on</strong> has to meet to be<br />

significant in the process of investment decisi<strong>on</strong> making and c<strong>on</strong>diti<strong>on</strong>s in which the evaluati<strong>on</strong> is<br />

made.<br />

The c<strong>on</strong>diti<strong>on</strong>s c<strong>on</strong>cern the intensity of the three main problems of investment evaluati<strong>on</strong>:<br />

indeterminacy, variability and immeasurability. The higher intensity of the problems the more<br />

difficult evaluati<strong>on</strong>.<br />

The specificity of IT investments evaluati<strong>on</strong> is that indeterminacy, variability and immeasurability<br />

appear in great force and their source is the nature of informati<strong>on</strong>, rapid IT development as well<br />

as the wide range and complexity of IT systems’ influence <strong>on</strong> organizati<strong>on</strong>s.<br />

The requirements and the c<strong>on</strong>diti<strong>on</strong>s of evaluati<strong>on</strong> determine the way in which the evaluati<strong>on</strong><br />

process should be realized: evaluati<strong>on</strong> criteria and measures, factors essential for evaluati<strong>on</strong>,<br />

activities necessary for the factors’ estimati<strong>on</strong> and methods of these activities realizati<strong>on</strong>.<br />

Perceiving evaluati<strong>on</strong> from system perspective:<br />

shows that selecti<strong>on</strong> of identificati<strong>on</strong> and quantificati<strong>on</strong> methods is as essential as selecti<strong>on</strong> of<br />

criteria and measures of the assessment, as evaluati<strong>on</strong> problems affect directly these two phases<br />

of evaluati<strong>on</strong> process;<br />

clarifies that decisi<strong>on</strong>s about acti<strong>on</strong>s carried out in the evaluati<strong>on</strong> process are taken by a<br />

compromise between the objective reflecti<strong>on</strong> of reality and usefulness in decisi<strong>on</strong> process;<br />

Shows that meta approaches to be useful in practice should take into account functi<strong>on</strong>ality<br />

criteri<strong>on</strong>.<br />

The classificati<strong>on</strong> of methods according to functi<strong>on</strong>ality into methods of identificati<strong>on</strong>, quantificati<strong>on</strong><br />

and assessment makes the selecti<strong>on</strong> process simpler. It allows avoiding mistakes such as perceiving<br />

methods which are of different roles as alternative <strong>on</strong>es or underrating methods of identificati<strong>on</strong> and<br />

quantificati<strong>on</strong> which could be of great importance in evaluati<strong>on</strong> of IT investments.<br />

References<br />

Ballantine, J. A., Galliers, R. B. and Stray, S. J. (1999) Informati<strong>on</strong> Systems/Technology Evaluati<strong>on</strong> Practices:<br />

Evidence from UK Organizati<strong>on</strong>s, in: L. Willcocks, S. Lester (eds.) Bey<strong>on</strong>d the IT Productivity Paradox:<br />

Assessment Issues, John Wiley & S<strong>on</strong>s, Chichester.<br />

Bannister, F. and Remenyi, D. (2000) Acts of Faith: Instinct, Value and IT Investment Decisi<strong>on</strong>s, Journal of<br />

Informati<strong>on</strong> Technology, Vol. 15, pp. 231-241.<br />

Berghout, E. and Renkema, T.-J. (2001) Methodologies for IT Investment Evaluati<strong>on</strong>: A Review and Assessment,<br />

in: W. Van Grembergen (ed.) Informati<strong>on</strong> Technology Evaluati<strong>on</strong> Methods and <strong>Management</strong>, Idea Group<br />

Publishing, Hershey.<br />

Botchkarev, A. and Andru, P. (2011) A Return <strong>on</strong> Investment as a Metric for Evaluating Informati<strong>on</strong> Systems:<br />

Tax<strong>on</strong>omy and Applicati<strong>on</strong>, Interdisciplinary Journal of Informati<strong>on</strong>, Knowledge, and <strong>Management</strong>, Vol. 6,<br />

pp. 245-269.<br />

Dos Santos, B. L. (1991) Justifying Investments in New Informati<strong>on</strong> Technologies, Journal of <strong>Management</strong><br />

Informati<strong>on</strong> Systems, Vol. 7, No 4, pp. 71-90.<br />

Farbey, B., Land, F. and Targett D. (1992) Evaluating Investments in IT, Journal of Informati<strong>on</strong> Technology, Vol.<br />

7 No. 2, pp. 109-122.<br />

Farbey, B., Land, F. and Targett, D. (1993) How to Assess Your IT Investment: a Study of Methods and Practice,<br />

Butterworth Heinemann, Oxford.<br />

Farbey, B., Land, F. and Targett, D. (1999) Evaluating Investments in IT: Finding a Framework, in: L. Willcocks,<br />

S. Lester (eds.) Bey<strong>on</strong>d the IT Productivity Paradox: Assessment Issues, John Wiley & S<strong>on</strong>s, Chichester.<br />

Jokela, P., Karlsudd, P. and Östlund, M. (2008) Theory, Method and Tools for Evaluati<strong>on</strong> Using a Systems-based<br />

Approach, The Electr<strong>on</strong>ic Journal Informati<strong>on</strong> Systems Evaluati<strong>on</strong>, Vol. 11 No 3, pp. 197 – 212, available<br />

<strong>on</strong>line at www.ejise.com.<br />

Jovanovic, B. and Rousseau, P. L. (2005) General Purpose Technologies, NBER Working Paper No. 11093,<br />

Cambridge.<br />

61


Jacek Cypryjanski<br />

Kambil, A., Henders<strong>on</strong>, J., Mohsenzadeh, H. (1993) Strategic <strong>Management</strong> of Informati<strong>on</strong> Technology<br />

Investment: An Opti<strong>on</strong>s Perspective, in: R. D. Banker, R. J. Kauffman, M. A. Mahmood (eds.) Strategic<br />

Informati<strong>on</strong> Technology <strong>Management</strong>: Perspectives <strong>on</strong> Organizati<strong>on</strong>al Growth and Competitive Advantage,<br />

Idea Group Publishing, Harrisburg.<br />

Kast, F. E. and Rosenzweig J. (1973) C<strong>on</strong>tingency Views of Organizati<strong>on</strong> and <strong>Management</strong>, Science Research<br />

Associates Inc. Publ., Palo Alto.<br />

Katz, D. and Kahn, R. L. (1980) The Definiti<strong>on</strong> and Identificati<strong>on</strong> of Organisati<strong>on</strong>s, in J. A. Litterer (ed.)<br />

Organizati<strong>on</strong>s, Structure and Behavior, 3rd. Edit.,John Wiley & S<strong>on</strong>s, NewYork.<br />

Kumar R. L. (1996) A Note <strong>on</strong> Project Risk and Opti<strong>on</strong> Values of Investments in Informati<strong>on</strong> Technologies,<br />

Journal of <strong>Management</strong> Informati<strong>on</strong> Systems, Vol. 13, No 1, pp. 187-193.<br />

Lee, Y.-Ch. and Lee, S.-S. (2011) The valuati<strong>on</strong> of RFID investment using fuzzy real opti<strong>on</strong>, Expert Systems with<br />

Applicati<strong>on</strong>s, Vol. 38, No 10, pp. 12195–12201.<br />

Lucas, Jr., H. C. (1993) The Business Value of Informati<strong>on</strong> Technology: A Historical Perspective and Thoughts<br />

for Future Research, in: R. D. Banker, R. J. Kauffman, M. A. Mahmood (eds.) Strategic Informati<strong>on</strong><br />

Technology <strong>Management</strong>: Perspectives <strong>on</strong> Organizati<strong>on</strong>al Growth and Competitive Advantage, Idea Group<br />

Publishing, Harrisburg.<br />

Mahmood, M. A. and Mann G. J. (2000) Special Issue: Impacts of Informati<strong>on</strong> Technology Investment <strong>on</strong><br />

Organizati<strong>on</strong>al Performance, Journal of <strong>Management</strong> Informati<strong>on</strong> Systems, Vol. 17 No. 1, pp. 3-10.<br />

Markus, M. L. and Soh C. (1993) Banking <strong>on</strong> the Informati<strong>on</strong> Technology: C<strong>on</strong>verting IT Spending into Firm<br />

Performance, in: R. D. Banker, R. J. Kauffman, M. A. Mahmood (eds.) Strategic Informati<strong>on</strong> Technology<br />

<strong>Management</strong>: Perspectives <strong>on</strong> Organizati<strong>on</strong>al Growth and Competitive Advantage, Idea Group Publishing,<br />

Harrisburg.<br />

Parker, M. M., Bens<strong>on</strong>, R. J. and Trainor, H. E. (1988) Informati<strong>on</strong> Ec<strong>on</strong>omics: Linking Business Performance to<br />

Informati<strong>on</strong> Technology, Prentice Hall, Englewood Cliffs.<br />

Parker, M. M., Trainor, H. E., Bens<strong>on</strong>, R. J. (1989) Informati<strong>on</strong> Strategy and Ec<strong>on</strong>omics: Linking Informati<strong>on</strong><br />

Systems Strategy to Business Performance, Prentice Hall, Englewood Cliffs.<br />

Peters, G. (1994) Evaluating your computer investment strategy, in: L. Willcocks (ed.) Informati<strong>on</strong> <strong>Management</strong>.<br />

The Evaluati<strong>on</strong> of Informati<strong>on</strong> Systems Investment, Chapman & Hall, L<strong>on</strong>d<strong>on</strong>.<br />

Peters, G. (1996) From Strategy to Implementati<strong>on</strong>: Identifying and Managing Benefits of IT investments, in: L.<br />

Willcocks (ed.) Informati<strong>on</strong> <strong>Management</strong>. The Evaluati<strong>on</strong> of Informati<strong>on</strong> Systems Investment, Chapman &<br />

Hall, L<strong>on</strong>d<strong>on</strong>.<br />

Powell, P. L. (1999) Evaluati<strong>on</strong> of Informati<strong>on</strong> Technology Investments: Business as Usual?, in: L. Willcocks, S.<br />

Lester (eds.) Bey<strong>on</strong>d the IT Productivity Paradox: Assessment Issues, John Wiley & S<strong>on</strong>s, Chichester.<br />

Remenyi, D., M<strong>on</strong>ey, A., Sherwood-Smith, M. and Irani, Z. (2000) Effective Measurement and <strong>Management</strong> of It<br />

Costs and Benefits, 2nd Edit., Butterworth-Heinemann, Woburn.<br />

Remenyi, D., Sherwood-Smith, M. and White, T. (1997) Achieving Maximum Value from Informati<strong>on</strong> Systems: A<br />

Process Approach, John Wiley & S<strong>on</strong>s, Chichester.<br />

Serafeimidis, V. (2001) A Review of Research Issues in Evaluati<strong>on</strong> of Informati<strong>on</strong> Systems, in: W. Van<br />

Grembergen (ed.) Informati<strong>on</strong> Technology Evaluati<strong>on</strong> Methods and <strong>Management</strong>, Idea Group Publishing,<br />

Hershey.<br />

Strassmann, P. A. (1997) The Squandered Computer: Evaluating the Business Alignment of Informati<strong>on</strong><br />

Technologies, The Informati<strong>on</strong> Ec<strong>on</strong>omics Press, New Canaan.<br />

Wen, H. J. and Sylla Ch. (1999) A Road Map for the Evaluati<strong>on</strong> of Informati<strong>on</strong> Technology Investment, in: M. A.<br />

Mahmood, E. J. Szewczak (eds.) Measuring Informati<strong>on</strong> Technology Investment Payoff: C<strong>on</strong>temporary<br />

Approaches, Idea Group Publishing, Hershey.<br />

Willcocks, L. (ed.) (1996) Investing in Informati<strong>on</strong> Systems: Evaluati<strong>on</strong> and Measurement, Chapman and Hall,<br />

L<strong>on</strong>d<strong>on</strong>.<br />

62


<strong>Academic</strong> Group and Forum <strong>on</strong> Facebook: Social, Serious<br />

Studies or Synergy?<br />

Ruth de Villiers and Marco Cobus Pretorius<br />

School of Computing, University of South Africa, Pretoria, South Africa<br />

Dvillmr@unisa.ac.za<br />

Marco.pretorius@gmail.com<br />

Abstract: An academic group and discussi<strong>on</strong> forum were established <strong>on</strong> Facebook for a cohort of postgraduate<br />

students studying the c<strong>on</strong>cepts and principles of eLearning. The Forum had a c<strong>on</strong>structivist, student-centric<br />

ethos, in which students initiated topics for discussi<strong>on</strong>, while the course leader and administrator facilitated.<br />

Previous research has been c<strong>on</strong>ducted, involving c<strong>on</strong>tent analysis of the topics and academic discourse, but the<br />

present study focuses <strong>on</strong> social aspects, investigating social- and study-related pursuits and determining whether<br />

synergy can exist between them. A literature review shows how social networking by students, initially social,<br />

began to overlap with academia, leading to the use of groups for academic purposes and forums for subjectrelated<br />

discussi<strong>on</strong>s. In the present study, data was triangulated and two methods of data analysis were used.<br />

Qualitative analysis was d<strong>on</strong>e <strong>on</strong> free-text data from students’ reflective essays to extract socially-related themes.<br />

Heuristic evaluati<strong>on</strong> was c<strong>on</strong>ducted by expert evaluators, who investigated forum discourse in line with<br />

c<strong>on</strong>temporary learning theory and who c<strong>on</strong>sidered the social culture of participati<strong>on</strong>. Findings of the qualitative<br />

analysis of students’ percepti<strong>on</strong>s and results of the heuristic evaluati<strong>on</strong> of forum participati<strong>on</strong> c<strong>on</strong>firmed each<br />

other, indicating a warm social climate and a c<strong>on</strong>ducive, well-facilitated envir<strong>on</strong>ment that supported individual<br />

participati<strong>on</strong> styles. It fostered inter-pers<strong>on</strong>al relati<strong>on</strong>ships between distance learners, as well as study-related<br />

relati<strong>on</strong>ships due to peer teaching and insights acquired from social negotiati<strong>on</strong>. The envir<strong>on</strong>ment supported<br />

student-initiative, but was moderated by facilitators. The mixed-methods research approach of evaluating<br />

students’ essays and c<strong>on</strong>ducting expert analysis of forum discussi<strong>on</strong>s showed the advent of a virtual community<br />

with a synergy between social aspects and academia. Most participants experienced a sound balance of social-<br />

and study-related benefits, but with a str<strong>on</strong>ger focus <strong>on</strong> academic matters.<br />

Keywords: eLearning, evaluati<strong>on</strong>, Facebook group, <strong>on</strong>line discussi<strong>on</strong> forums, qualitative analysis<br />

1. Introducti<strong>on</strong><br />

Social networking sites (SNS’s) are increasingly used in academia. This paper discusses the social<br />

climate of an academic group and <strong>on</strong>line discussi<strong>on</strong> forum (ODF) established <strong>on</strong> the SNS, Facebook,<br />

to enhance learning for postgraduate distance-learners studying ‘C<strong>on</strong>cepts and Principles of<br />

eLearning’ at the University of South Africa (UNISA). Most of the students were professi<strong>on</strong>als.<br />

ODF’s are a comm<strong>on</strong> feature in web-based groups and eLearning envir<strong>on</strong>ments. UNISA, a distanceeducati<strong>on</strong><br />

instituti<strong>on</strong>, provides ODF’s <strong>on</strong> its official site, but we offered an alternative supplementary<br />

group and discussi<strong>on</strong> forum <strong>on</strong> Facebook for a postgraduate cohort. It had a c<strong>on</strong>structivist, studentcentric<br />

nature, in which students pers<strong>on</strong>ally initiated the discussi<strong>on</strong> topics. The course leader and an<br />

administrator facilitated as guides <strong>on</strong> the side, rather than as sages <strong>on</strong> the stage. The aim was to<br />

encourage interacti<strong>on</strong> that provided subject-related informati<strong>on</strong> and academic discourse. Early<br />

research about the Group (de Villiers, 2010) involved c<strong>on</strong>tent analysis of the topics and discussi<strong>on</strong>s,<br />

using quantitative frequency counts of interacti<strong>on</strong> types, and qualitative discourse analysis to<br />

investigate the academic c<strong>on</strong>tent. The study showed that active participati<strong>on</strong> in the Forum supported<br />

learning and enhanced performance. Sec<strong>on</strong>dary benefits also occurred, including the emergence of<br />

peer-to-peer relati<strong>on</strong>ships. The present study therefore focuses <strong>on</strong> social aspects of the Group.<br />

2. Literature review<br />

Various studies have addressed students’ use of SNS’s and determined that interacti<strong>on</strong>s were<br />

primarily social (Madge, Meek, Wellens and Hooley, 2009; Selwyn, 2009). In an <strong>on</strong>line survey of 600<br />

student users, Mazman and Usluel (2010) found they visited Facebook for approximately 30 minutes<br />

daily, mainly for socializing. A meta-analysis of 36 studies <strong>on</strong> students’ and teachers’ use of<br />

Facebook, indicates little educati<strong>on</strong>al use (Hew, 2011). According to Lampe, Ellis<strong>on</strong> and Steinfield<br />

(2008), Facebook is ubiquitous <strong>on</strong> US campuses with the typical user visiting for 80 minutes daily.<br />

However, Lampe et al found that academic matters such as lectures, reading materials, deliverables,<br />

and instructors were menti<strong>on</strong>ed and about 15% of students used Facebook to c<strong>on</strong>tact lecturers.<br />

Selwyn (2009) studied Facebook Walls of UK undergraduates, visiting over 600 sites with public<br />

viewing profiles. Four percent of the exchanges related to academic schedules, venues, lectures and<br />

deliverables, while another theme was criticism of keen students, seminars and lecturers.<br />

63


Ruth de Villiers and Marco Cobus Pretorius<br />

Nevertheless, educati<strong>on</strong>al use is <strong>on</strong> the increase and explicit academic use of SNS’s is reported. Four<br />

case studies <strong>on</strong> social networking by students (J<strong>on</strong>es, Blackey, Fitzgibb<strong>on</strong> and Chew, 2010) show a<br />

divide between students’ learning space and pers<strong>on</strong>al space, yet acknowledge that educators should<br />

leverage SNS’s and create envir<strong>on</strong>ments for independent learning, reflecti<strong>on</strong>, and communities of<br />

inquiry. Mazman and Usluel (2010) define educati<strong>on</strong>al use of Facebook as involving communicati<strong>on</strong><br />

(discussi<strong>on</strong>s and informati<strong>on</strong>); collaborati<strong>on</strong> in groups; and resource sharing via videos and links.<br />

There is increasing academic use of Facebook in South Africa, the home base of the authors. Bosch<br />

(2009:147) did a ‘virtual ethnographic’ study of Facebook profiles of 200 students, supplemented by<br />

interviews with students and staff who communicated <strong>on</strong> Facebook and found that the experience<br />

undid ‘traditi<strong>on</strong>al power hierarchies’. Students were more engaged <strong>on</strong> Facebook than <strong>on</strong> the official<br />

course management site. Many bel<strong>on</strong>ged to groups for societies and academic programmes, where<br />

they shared resources and logistical informati<strong>on</strong> and checked class-related material. Visagie and de<br />

Villiers (2010) (not the present author) surveyed 32 academics and established that 56% of them<br />

would c<strong>on</strong>sider using Facebook as an academic tool. As academic use of Facebook increases,<br />

research is being c<strong>on</strong>ducted <strong>on</strong> subject-related discourse in groups and communities. The primary<br />

author of this paper did detailed analysis of the academic c<strong>on</strong>tent of the discourse <strong>on</strong> a postgraduate<br />

discussi<strong>on</strong> forum (de Villiers, 2010). First-year students participated in peer-initiated topic-based<br />

c<strong>on</strong>versati<strong>on</strong>s in a systematic and well-articulated way (Rambe and Ng’ambi, 2011). Informal learning<br />

occurred in a social-c<strong>on</strong>structivist community where students and instructors c<strong>on</strong>versed and shared<br />

knowledge to help each other understand the subject matter better (Ractham and Firpo, 2011).<br />

3. Research design and methods<br />

The research questi<strong>on</strong> addressed in this study is:<br />

Did the venture serve both social- and study-related pursuits in a synergistic manner?<br />

To evaluate whether academic forums <strong>on</strong> Facebook can have a synergistic value, we focused <strong>on</strong> the<br />

social culture and interacti<strong>on</strong> patterns described by students in reflective essays and identified by<br />

heuristic evaluators studying the discussi<strong>on</strong>s. We investigated whether the Forum supported pers<strong>on</strong>al<br />

participati<strong>on</strong> styles and valuable interacti<strong>on</strong>. The t<strong>on</strong>e, nature, impact and facilitati<strong>on</strong> of the<br />

discussi<strong>on</strong>s were c<strong>on</strong>sidered, as well as the ethos of the community. This study evaluates the<br />

integrati<strong>on</strong> of social aspects and serious studies, by using a mixed-methods research approach<br />

(Creswell, 2009), which was triangulated by two evaluati<strong>on</strong> methods and two different datasets:<br />

Study 1 Qualitative analysis of free-text data from students’ reflective essays: These percepti<strong>on</strong>s<br />

were qualitatively analysed using a form of grounded theory.<br />

Study 2 Heuristic evaluati<strong>on</strong> (HE) by four expert evaluators: These experts investigated forum<br />

c<strong>on</strong>tributi<strong>on</strong>s to determine the social climate and to establish whether the ethos of the discussi<strong>on</strong>s<br />

c<strong>on</strong>formed to c<strong>on</strong>temporary eLearning theories. These evaluati<strong>on</strong>s were mainly quantitative.<br />

The research incorporated data triangulati<strong>on</strong> as both essays and forum discourse were analysed, and<br />

methodological triangulati<strong>on</strong> via the combinati<strong>on</strong> of qualitative analysis and heuristic evaluati<strong>on</strong>. For<br />

ethical reas<strong>on</strong>s participants were informed that research was being c<strong>on</strong>ducted, and they signed<br />

informed c<strong>on</strong>sent forms.<br />

4. Study 1: Qualitative analysis of free-text data from students’ reflective<br />

essays<br />

Membership of the Group was encouraged, but not compulsory. Thirty of the 40 in the cohort joined.<br />

Twenty seven completed the course, including 21 Group members, twelve of whom were very active<br />

<strong>on</strong> the Forum. Textual data from the students’ reflective essays was analysed by grounded theory<br />

and categorised under themes and sub-themes that emerged. The findings are presented and<br />

supported by groups of quotati<strong>on</strong>s from students’ essays, in the students’ words. The students whose<br />

reflecti<strong>on</strong>s are quoted, are cited, e.g. P1 represents Participant 1 and NP1 is N<strong>on</strong>-Participant 1.<br />

4.1 Social vibes and ethos of the virtual community<br />

In off-forum essays, students gave varying perspectives <strong>on</strong> the ethos and impact of the discourse.<br />

64


Virtuality became Reality<br />

Ruth de Villiers and Marco Cobus Pretorius<br />

Distance dissipated, as participants got ‘to know each other’ and c<strong>on</strong>versed ‘in the presence of peers’:<br />

‘Since it is often outside the classroom that students get to real knowledge sharing, social<br />

networks can play a major role in informal social learning, giving access to each others’<br />

implicit knowledge’.<br />

‘The group are friendly, enthusiastic and passi<strong>on</strong>ate about the subject... the interacti<strong>on</strong> is<br />

excellent’.<br />

The group became a community and had a sense of real-world talking and listening:<br />

‘If I share my thoughts, I like to know who is listening. It is gratifying to interact with<br />

people with a comm<strong>on</strong> goal’.<br />

‘What you teach fellow students is embedded in your mind l<strong>on</strong>ger... because you say it to<br />

people’<br />

(P3, P5, P12, P14, P17).<br />

Culture and t<strong>on</strong>e of interacti<strong>on</strong>s<br />

The envir<strong>on</strong>ment was warm and c<strong>on</strong>ducive to discussi<strong>on</strong>s. Some students built relati<strong>on</strong>ships and<br />

c<strong>on</strong>versed off-forum. Although real-world academia can be self-focused and competitive, the Group<br />

culture was not geared to pers<strong>on</strong>al achievement:<br />

‘The best is that users freely share their sources of informati<strong>on</strong>, how they interpret<br />

c<strong>on</strong>cepts, and their pers<strong>on</strong>al experiences’.<br />

‘Wisdom of crowds... the whole is greater than the parts’.<br />

‘Make or break depends <strong>on</strong> support of peers... those with experience and intent to help<br />

can mentor and guide novices in social networking’.<br />

‘I enjoyed interacting with fellow students <strong>on</strong> a social level, although I did not benefit<br />

academically’.<br />

‘Interacti<strong>on</strong> was enjoyable and fulfilling’.<br />

‘I have (previously) used forums to pose questi<strong>on</strong>s and seek answers, but I disliked the<br />

dull, standardized and uncreative way they were presented’.<br />

The t<strong>on</strong>e was informal yet cordial: ‘Nobody addressed the lecturer as “Madame”, yet <strong>on</strong> the other<br />

hand, there was no use of shortcuts like B4, 2b or LOL’ (P4, P6, P7, P13, P16, P27).<br />

Removal of isolati<strong>on</strong><br />

‘Standard, boring distanceLearning was enriched’; ‘The sense of isolati<strong>on</strong> dissipated’.<br />

To a l<strong>on</strong>g-term distance-learner, it was ‘a thrilling experience. The first exotic e-fun occurred when<br />

fellow-students introduced themselves as if sitting al<strong>on</strong>gside me, but writing from Australia, Japan,<br />

Namibia, Pretoria’ (P1, P4, P12).<br />

Challenge, yet affirmati<strong>on</strong><br />

Written words do not vaporise like spoken words and this calls for careful reflecti<strong>on</strong> before posting:<br />

‘... a new-found sense of pressure to understand what I was reading’.<br />

‘Some<strong>on</strong>e else was going to be reading it, and giving their opini<strong>on</strong>’ (P13, P21).<br />

However, c<strong>on</strong>tributing brought affirmati<strong>on</strong>: ‘The brief experience when I shared my views was a<br />

turning point...My c<strong>on</strong>fidence peaked...’.<br />

‘It was heartening to see that a number of fellow-students agreed... ’.<br />

‘Being introvert, I <strong>on</strong>ly made <strong>on</strong>e comment, but it is a start’.<br />

‘Participating with the professor and fellow students, I felt h<strong>on</strong>oured to be part of the<br />

exercise and especially getting accolades for my c<strong>on</strong>tributi<strong>on</strong>s’ (P2, P25, P27).<br />

65


Ruth de Villiers and Marco Cobus Pretorius<br />

4.2 Support for individual styles of participati<strong>on</strong><br />

Most students found the Group and Forum supportive. Several exercised pro-active leadership and<br />

initiated topics, while others saw it as a place to participate in discussi<strong>on</strong>s without the exposure of<br />

c<strong>on</strong>tact-learning. Yet others chose not to c<strong>on</strong>tribute, but observed and ‘listened’. They benefitted,<br />

although some fellow-learners did not appreciate having observers. While some students found the<br />

atmosphere of the Forum to be n<strong>on</strong>-intimidating, other tended to be daunted:<br />

Better than a real classroom<br />

Some participants were more comfortable c<strong>on</strong>versing <strong>on</strong> Facebook than in a c<strong>on</strong>venti<strong>on</strong>al classroom:<br />

‘Sharing is less rigid than when resp<strong>on</strong>ding to questi<strong>on</strong>s in a class’.<br />

Learners are ‘not frowned <strong>on</strong> when they express themselves in whatever way they feel<br />

comfortable’.<br />

‘People are less afraid, and speak without fear of being mocked’ (P7, P17).<br />

‘It eliminated possible first-line prejudices that might have occurred in a c<strong>on</strong>tact situati<strong>on</strong>.<br />

One c<strong>on</strong>sidered the c<strong>on</strong>tent and not the pers<strong>on</strong>’ (P21).<br />

‘Got more feedback than in class situati<strong>on</strong>s, where a few students may dominate’ (P4).<br />

N<strong>on</strong>-intimidating – a place to take initiative<br />

The ethos encouraged some members to be forthcoming:<br />

‘Should I take the initiative?...it was clear this was a place to take charge... Grasping the<br />

new-found freedom, I decided to start...’.<br />

‘Some are outspoken and involved in everything, with quick resp<strong>on</strong>ses, but others keep<br />

to themselves’.<br />

We could ‘...have a brainstorm sessi<strong>on</strong>’ (P7, P13, P17).<br />

Daunting<br />

Some felt intimidated and inadequate. They feared negative resp<strong>on</strong>ses or no resp<strong>on</strong>se:<br />

Exposure to ‘...some refined and polished c<strong>on</strong>tributi<strong>on</strong>s, led to feelings of academic inferiority’.<br />

‘People can be afraid to express views, because they are unsure of relevance and<br />

accuracy’.<br />

‘You would like to c<strong>on</strong>tribute or ask questi<strong>on</strong>s, but w<strong>on</strong>der if you will look stupid’ (P5,<br />

P10, P21).<br />

Resp<strong>on</strong>ses to postings<br />

C<strong>on</strong>tributors anticipated resp<strong>on</strong>ses to their postings and were disappointed if this did not happen fast:<br />

‘You are demoralised if no feedback is forthcoming’.<br />

‘The time-independent nature of the interacti<strong>on</strong>s meant that discussi<strong>on</strong>s were sometimes<br />

drawn out, preventing immediate feedback...’.<br />

‘You (have to) wait for the resp<strong>on</strong>se when somebody is <strong>on</strong>line’ (P8, P16, P21).<br />

Observers<br />

Some perceived the Group as a safe space for learning without c<strong>on</strong>tributing. They chose to learn by<br />

watching and listening, yet without the negative c<strong>on</strong>notati<strong>on</strong>s of ‘lurking’. Certain participants were<br />

disturbed by these <strong>on</strong>lookers:<br />

‘I experienced frustrati<strong>on</strong> when just a few participants c<strong>on</strong>tributed, though I realise that<br />

some preferred to read what others wrote rather than c<strong>on</strong>tributing’.<br />

‘Some joined the group, but did not make any postings’.<br />

‘Some students joined but kept silent... just watching, a bit creepy!’ (P1, P17, P27).<br />

66


Ruth de Villiers and Marco Cobus Pretorius<br />

P25, who was an observer, explained, ‘I go <strong>on</strong> forum to see if some<strong>on</strong>e asks what I want to know. It<br />

helps me know if I am going in the correct directi<strong>on</strong>. I log in daily and am disappointed when there are<br />

no new c<strong>on</strong>tributi<strong>on</strong>s’.<br />

4.3 Academia <strong>on</strong> Web 2.0 and Facebook<br />

On Web 1.0 and via eLearning 1.0, learners access existing educati<strong>on</strong>al Web c<strong>on</strong>tent. In c<strong>on</strong>trast,<br />

Web 2.0 and eLearning 2.0 (Ebner, 2007) empower learners to pers<strong>on</strong>ally c<strong>on</strong>tribute c<strong>on</strong>tent.<br />

Web 2.0 and a paradigm shift<br />

In the c<strong>on</strong>text of educati<strong>on</strong> ‘Web 2.0 means a learner-centric approach’ (P16).<br />

More a ‘social revoluti<strong>on</strong>’/ ‘social phenomen<strong>on</strong>’ than a technological issue.<br />

‘...a paradigm shift... we need to relinquish tried and tested ways, which takes time and<br />

not every<strong>on</strong>e joined the revoluti<strong>on</strong>’ (P7, P8, P16).<br />

Some could be ‘even more sociable than before’ but ‘others are just not sociable’ (P12).<br />

Several cited Ebner (2007), ’Technical issues will be solved quickly, but to change the<br />

thinking about learning and teaching is hard and l<strong>on</strong>g’.<br />

‘We can’t expect every<strong>on</strong>e to feel comfortable with social tools, but change is a<br />

c<strong>on</strong>stant...’ (P3, P11).<br />

Academia <strong>on</strong> Facebook<br />

Some were c<strong>on</strong>vinced that this was the way forward:<br />

‘For some<strong>on</strong>e like me, who already uses Facebook and enjoys working smartly, Fb<br />

provides a single point of entry from which I interact socially, stay up to date, and<br />

participate in communities... I am comfortable using it as a learning tool’.<br />

‘This type of discussi<strong>on</strong> forum works with what is already available’.<br />

‘We are the mobile-interc<strong>on</strong>nected-global-village generati<strong>on</strong> with Web 2.0 Fever’ (P7,<br />

P13).<br />

Furthermore, Facebook is ideal for forums. ‘It is well-structured with predefined areas for discussi<strong>on</strong>s,<br />

the Wall for banter, pictures and videos, membership lists, ways to handle permissi<strong>on</strong> and access’;<br />

‘...has global interc<strong>on</strong>necti<strong>on</strong>’ and ‘You can reach members pers<strong>on</strong>ally by accessing<br />

profile pages’. ‘...a co-operative envir<strong>on</strong>ment that fosters trust am<strong>on</strong>g learners and<br />

instructor’, allowing students to ‘learn from <strong>on</strong>e another’ (P7, P11, P13, P18).<br />

P11 made a str<strong>on</strong>g statement: ‘Educati<strong>on</strong>al instituti<strong>on</strong>s should use Facebook for learning and make<br />

links from their instituti<strong>on</strong>al websites‘.<br />

Shy users and silent users<br />

Facebook breaks barriers for those who are shy or who feel vulnerable. Despite being a public space,<br />

it provides c<strong>on</strong>cealment that shields members when they pass opini<strong>on</strong>s:<br />

Those who ‘struggle to socialise’ or ‘have difficulty with social skills’ found it easier to communicate <strong>on</strong><br />

Facebook than face-to-face. ‘Collaborative <strong>on</strong>line learning brings major changes, so that learners with<br />

low self-esteem can communicate and comment without physical interacti<strong>on</strong>’ (P4, P8, P10).<br />

Then there were the silent observers – see Secti<strong>on</strong> 4.2. Some were insecure and chose not to<br />

communicate, but essays indicated that others had indeed joined, but were unable to access the<br />

Forum. At least two of them wrote <strong>on</strong> the Wall, but did not find out how to take part in discussi<strong>on</strong>s until<br />

it was too late (P4, P5). The usability of Facebook and access to Groups has since been improved.<br />

67


Asynchr<strong>on</strong>icity and synchr<strong>on</strong>icity<br />

Ruth de Villiers and Marco Cobus Pretorius<br />

Asynchr<strong>on</strong>ous interacti<strong>on</strong> via social networking offers Ebner’s (2007) ‘Triple A Factor: anytime,<br />

anywhere, any<strong>on</strong>e’, regardless of locati<strong>on</strong> and time (P3). Opini<strong>on</strong>s varied <strong>on</strong> asynchr<strong>on</strong>icity:<br />

Some appreciated that ‘questi<strong>on</strong>s and answers could be carefully thought out before posting’,<br />

whereas others felt that it ‘detracted from sp<strong>on</strong>taneity... debate is interrupted by time lapses’ or<br />

c<strong>on</strong>versely that ‘it moved fast, I struggled to c<strong>on</strong>tribute’. Furthermore, asynchr<strong>on</strong>icity results in<br />

different threads and ‘... at times, it was difficult to follow them all’ (P1, P17, P21).<br />

There was little use of synchr<strong>on</strong>ous Chats, although some learners held small-group c<strong>on</strong>versati<strong>on</strong>s in<br />

real time.<br />

N<strong>on</strong>-compulsory membership<br />

Several participants would have liked membership to be mandatory, but the course leader took a<br />

c<strong>on</strong>sidered decisi<strong>on</strong> not to enforce it. A high achiever who chose not to join, explained why she<br />

appreciated the flexibility: ‘I have a solitary, intrapers<strong>on</strong>al and introspective learning style. I p<strong>on</strong>der<br />

and evaluate, and write down thoughts... I tackle problems and soluti<strong>on</strong>s al<strong>on</strong>e’ (NP1).<br />

4.4 C<strong>on</strong>trol and management by the facilitators<br />

<strong>Management</strong> of the forum was challenging. Since the explicit ethos was student-initiati<strong>on</strong> of<br />

discussi<strong>on</strong>s, we positi<strong>on</strong>ed ourselves as facilitators between the extremes of str<strong>on</strong>g c<strong>on</strong>trol and<br />

hands-off. We served as guides <strong>on</strong> the side, not as sages <strong>on</strong> the stage. <strong>Management</strong> involved<br />

carefully watching accuracy of the c<strong>on</strong>tent, as well as m<strong>on</strong>itoring security.<br />

Security<br />

Some students felt threatened by security breaches: ‘It is difficult for me to use Facebook socially, let<br />

al<strong>on</strong>e as a learning tool. My reservati<strong>on</strong>s are due to lack of security...’ (P11).<br />

We err<strong>on</strong>eously admitted an intruder, believing he was a student whose registrati<strong>on</strong> was not finalised.<br />

He participated, then posted advertisements for motivati<strong>on</strong>al courses and financial products! A<br />

disc<strong>on</strong>certed student unveiled him when they communicated off-forum and she challenged him with<br />

an academic questi<strong>on</strong> he could not answer: ‘It is exciting to say I have encountered an e-stalker! Yet I<br />

must questi<strong>on</strong> how he managed to infiltrate our group’ (P1). As facilitators, we immediately removed<br />

him. The shrewd P1 picked up another anomaly: ‘A profile image introduced a beautiful young lady<br />

and we chatted away <strong>on</strong> academic matters. Her achievements amazed me. After a few weeks she<br />

admitted to being “he”, an older student, who had borrowed his daughter’s Facebook membership<br />

(with Prof’s permissi<strong>on</strong>) due to logistical difficulties’.<br />

C<strong>on</strong>trol, please!<br />

Some students wanted tighter management:<br />

‘Such platforms need proper c<strong>on</strong>trol and facilitati<strong>on</strong>’; ‘All c<strong>on</strong>tent should be verified’.<br />

‘People should not be allowed to say just anything – there was irrelevant c<strong>on</strong>tent <strong>on</strong> the<br />

Wall’ (the intruder).<br />

‘Without verificati<strong>on</strong> or pers<strong>on</strong>al discernment, learners could be misled by inaccurate<br />

statements’.<br />

‘A weekly questi<strong>on</strong> from the facilitators might have encouraged more interacti<strong>on</strong>’.<br />

Another queried whether a discussi<strong>on</strong> forum could be effective without ‘central guidance from a<br />

lecturer or teacher’ (P3, P4, P12, P17).<br />

C<strong>on</strong>structivism<br />

Others appreciated the c<strong>on</strong>structivist-style freedom and low-level c<strong>on</strong>trol:<br />

The Forum was an implementati<strong>on</strong> of the current ‘focus <strong>on</strong> cognitivism and c<strong>on</strong>structivism’.<br />

68


Ruth de Villiers and Marco Cobus Pretorius<br />

‘Well moderated; well managed’.<br />

‘A new paradigm of teacher-learner interacti<strong>on</strong>. The course leader merely facilitates and<br />

guides’.<br />

It was ‘not dictated by the teacher’ who is adapting ‘to new ways of teaching and guiding’.<br />

It could have been managed ‘by fixed principles, but that would curtail the c<strong>on</strong>versati<strong>on</strong> – which was<br />

not the idea behind this free, natural learning interacti<strong>on</strong>’.<br />

‘The input snippets received from the leader and administrator are gold nuggets’.<br />

(P1, P2, P7, P8, P16. P18).<br />

Reliability and validity<br />

Members and a n<strong>on</strong>-member expressed c<strong>on</strong>cern about how to distinguish between fact and the<br />

opini<strong>on</strong>s of peers: There was ‘potential both to c<strong>on</strong>fuse and illuminate, c<strong>on</strong>fusing when it’s a collecti<strong>on</strong><br />

of “I think...” without proper backing. However, when the posts are well thought through and backed<br />

with credible references, the potential for real learning is high’ (P21).<br />

‘Teachers should set standards and test c<strong>on</strong>tributi<strong>on</strong>s before they are posted <strong>on</strong> the site’<br />

(P3).<br />

Our resp<strong>on</strong>se to this is, first, that pre-approval is infeasible in forums and, sec<strong>on</strong>d, as facilitators, we<br />

were loath to destroy sp<strong>on</strong>taneity. M<strong>on</strong>itoring must be d<strong>on</strong>e after postings, and be handled with<br />

discerning public comments and private communicati<strong>on</strong> with offenders. If c<strong>on</strong>tent was merely weak,<br />

we did not react, but <strong>on</strong> <strong>on</strong>e occasi<strong>on</strong> when discourse veered off-track, the course leader resp<strong>on</strong>ded<br />

by pointing to theory. This c<strong>on</strong>cern may have been a reas<strong>on</strong> why some n<strong>on</strong>-participants did not join<br />

the Group: ‘What proves that the points shared by a student are true and valid?’ (NP2).<br />

4.5 Balancing academic and social interacti<strong>on</strong>s<br />

To users accustomed to using SNS’s for entertainment, the playing fields now offered study facilities!<br />

Successful integrati<strong>on</strong> of social and serious<br />

The Wall and Introduce Yourself provided informality. They offset the distance and set a<br />

friendly c<strong>on</strong>text for the study-related pursuits. Most members felt that social networking<br />

and serious studies could be effectively combined. Several menti<strong>on</strong>ed the incorporati<strong>on</strong><br />

of ‘fun’, ‘entertainment’, ‘informality’, ‘interactivity’ into learning (P4, P7, P8, P14, P16,<br />

P17, P21).<br />

‘I definitely recommend eLearning via Facebook’.<br />

‘I learned to melt into social networking scenes, let the resistance go, flow with the wave,<br />

yet keep wearing the academic hat’.<br />

‘Push and pull factors: friends pull; academia pushed us to view Facebook as a serious<br />

tool’.<br />

‘Social and educati<strong>on</strong>al tasks are executed simultaneously. I peep at the study group site<br />

each time I log <strong>on</strong>’ (P1, P7, P10).<br />

Given their ubiquity, it would be ‘short-sighted to ignore Web 2.0 applicati<strong>on</strong>s’ for educati<strong>on</strong>al<br />

purposes’ (P21, citing Ebner, 2007).<br />

Even a n<strong>on</strong>-participant commented, ’Facebook has caused addicti<strong>on</strong>... a study group there could be a<br />

good way to study’ (NP6).<br />

Distracti<strong>on</strong>s<br />

Some struggled with distracti<strong>on</strong>s:<br />

‘Other Facebook interacti<strong>on</strong>s and the whole Internet could easily pull <strong>on</strong>e away...’.<br />

‘ ...numerous inviting sites could attract learners to something totally different’.<br />

‘Family and friends found me and nagged to be my friend‘; ’...friends determined to<br />

“poke” me’.<br />

69


Ruth de Villiers and Marco Cobus Pretorius<br />

‘It calls for a change in mindset am<strong>on</strong>g those who see it as a fun tool and miss its<br />

essence in learning’ (P1, P16, P17).<br />

Potential distracti<strong>on</strong> was a reas<strong>on</strong> why <strong>on</strong>e n<strong>on</strong>-participant did not join. ‘Literature indicates that SNS<br />

tools and systems incorporate high interactivity to keep users interested. This could distract from<br />

learning’ (NP2).<br />

4.6 Nature of discourse and debate<br />

Simulated face-to-face discussi<strong>on</strong>s were enriching for distance learners:<br />

Interactive communicati<strong>on</strong> between peers<br />

‘The ability to interact with people of similar interests from anywhere in the world, was a<br />

definite advantage. One could tap into the collective c<strong>on</strong>sciousness of a diverse group of<br />

people’.<br />

‘We are exposed to having views challenged and can engage in discussi<strong>on</strong>s of the<br />

subject matter’.<br />

‘Opini<strong>on</strong>s differ over same material, but without challenging others disrespectfully’.<br />

‘Different perspectives <strong>on</strong> the same topic...’ (P2, P7, P10, P21).<br />

New insights<br />

Students learned from their peers and it is significant that the more active participants all performed<br />

well in the examinati<strong>on</strong>. Matters emerged that learners had not identified independently:<br />

‘Collectively the learners are exposed to an abundance of informati<strong>on</strong>... collaboratively<br />

they digest c<strong>on</strong>tent and informati<strong>on</strong> within a short time’.<br />

‘...useful perspectives, bey<strong>on</strong>d what <strong>on</strong>e would obtain by merely reading the articles’.<br />

‘The whole community benefits from <strong>on</strong>e anothers’ insights’.<br />

‘Current informati<strong>on</strong> and state-of-the art development make a significant c<strong>on</strong>tributi<strong>on</strong> to<br />

learning’.<br />

And a perceptive point made by different students:<br />

‘By posting ideas, we solidify our thoughts. By reading others’ resp<strong>on</strong>ses, our ideas are<br />

refined’.<br />

‘I gained insight through reading posts of others, and the process of thinking through my<br />

resp<strong>on</strong>ses helped clarify issues’. ‘When reading fellow students’ input..., my own<br />

interpretati<strong>on</strong> changed’<br />

(P2, P10, P16, P21, P25).<br />

Generati<strong>on</strong>al differences<br />

Percepti<strong>on</strong>s and approaches deferred. Some older students joined Facebook as novices and became<br />

avid c<strong>on</strong>tributors. Three participants menti<strong>on</strong>ed their need to print the discourse, while some students<br />

from the Net-generati<strong>on</strong> preferred the e-word to the printed or spoken word:<br />

‘The ability to recall and regain <strong>on</strong>line discussi<strong>on</strong>s is vastly superior to n<strong>on</strong>-eLearning<br />

scenarios of searching through paper-based materials or trying to recall verbal<br />

c<strong>on</strong>versati<strong>on</strong>s’ (P21).<br />

‘I view Facebook as a purely social tool for the younger generati<strong>on</strong> and unsuitable for<br />

academic purposes. It was a novel approach, but should have just been an experiment...’<br />

(P27).<br />

5. Study 2: Heuristic evaluati<strong>on</strong> by expert evaluators<br />

Four expert evaluators, who are profiled in Table 1, c<strong>on</strong>ducted a heuristic evaluati<strong>on</strong> (HE) to<br />

investigate the social climate of the Forum and to assess c<strong>on</strong>tributi<strong>on</strong>s against c<strong>on</strong>temporary<br />

eLearning theories that are based <strong>on</strong> human-centred values. In order to do the evaluati<strong>on</strong>, they<br />

c<strong>on</strong>sidered sets of criteria (also termed heuristics) to establish whether the discussi<strong>on</strong>s c<strong>on</strong>formed to<br />

70


Ruth de Villiers and Marco Cobus Pretorius<br />

the pedagogies associated with c<strong>on</strong>structivism, customizati<strong>on</strong> and creativity, as well as judging the<br />

social aspects of the experience.<br />

Table 1: Profiles of the expert evaluators<br />

Evaluator Occupati<strong>on</strong> Expertise Involvement in Group<br />

A Researcher Evaluati<strong>on</strong>; ELearning;<br />

heuristic evaluati<strong>on</strong> (HE)<br />

No involvement<br />

B Lecturer and post- ELearning envir<strong>on</strong>ments; Member and active<br />

graduate student<br />

HE<br />

c<strong>on</strong>tributor<br />

C Usability<br />

Usability evaluati<strong>on</strong>;<br />

Administrator<br />

practiti<strong>on</strong>er ELearning websites; HE<br />

D IT professor ELearning; HE;<br />

Human-computer interacti<strong>on</strong>.<br />

No involvement<br />

Each expert evaluator performed his/her evaluati<strong>on</strong> independently. All four were ‘double experts’,<br />

namely experts both in eLearning and in heuristic evaluati<strong>on</strong>. One was a student who had been a<br />

member of the Group and another was the Group administrator. The evaluators did not see the offforum<br />

reflective essays (Study 1 data), but c<strong>on</strong>sidered and evaluated the discourse <strong>on</strong> the Forum.<br />

The evaluati<strong>on</strong> template comprised four sets of criteria, twelve criteria in total, phrased as evaluati<strong>on</strong><br />

statements and rated <strong>on</strong> a 5 to 1 Likert scale, where 5 was Str<strong>on</strong>gly agree and 1 was Agree. There<br />

were also spaces for evaluators to provide open-ended comments.<br />

Table 2 tabulates the criteria against the quantitative results and is followed by a discussi<strong>on</strong> <strong>on</strong> the<br />

four factors evaluated.<br />

Table 2: Results of the heuristic evaluati<strong>on</strong><br />

Str<strong>on</strong>gly agree<br />

(5)<br />

Agree (4) Neutral (3) Disagree (2) Str<strong>on</strong>gly disagree (1)<br />

Category and criteria – rated <strong>on</strong> the scale above Average rating<br />

1. C<strong>on</strong>structivism<br />

The activities undertaken in the Group are highly c<strong>on</strong>structivist. 3.5<br />

Participants in discussi<strong>on</strong>s think independently and make pers<strong>on</strong>al interpretati<strong>on</strong>s. 4.125<br />

Discussi<strong>on</strong>s moved bey<strong>on</strong>d the curriculum and applied c<strong>on</strong>cepts in the real world. 4.75<br />

Cross-criteri<strong>on</strong> average<br />

2. Customisati<strong>on</strong><br />

4.125<br />

Participants can customise the time and place of their interactive learning. 5.0<br />

The discussi<strong>on</strong> forum is learner-centric in that participants could select and initiate<br />

their own topics for discussi<strong>on</strong> and could c<strong>on</strong>tribute pers<strong>on</strong>al c<strong>on</strong>tent.<br />

5.0<br />

Cross-criteri<strong>on</strong> average 5.0<br />

3. Creativity<br />

<strong>Academic</strong> discussi<strong>on</strong>s in the Forum represent an innovative way of using Facebook<br />

for learning purposes.<br />

Participants resp<strong>on</strong>ded to the Group envir<strong>on</strong>ment in creative ways. 4.25<br />

Cross-criteri<strong>on</strong> average 4.625<br />

4. Social climate of the Group<br />

Interacti<strong>on</strong> <strong>on</strong> the Forum took place in a friendly and c<strong>on</strong>ducive envir<strong>on</strong>ment. 4.5<br />

The distance learners who joined the Group got to ‘know’ each other. 4.5<br />

The ethos of the Forum supported individual styles of participati<strong>on</strong>. 4.0<br />

Cross-criteri<strong>on</strong> average 4.33<br />

The way the Forum was managed, resulted in a space that was:<br />

rigid/strictly<br />

c<strong>on</strong>trolled<br />

(1)<br />

firmly<br />

c<strong>on</strong>trolled<br />

(2)<br />

balanced and<br />

well<br />

moderated<br />

(3)<br />

led by<br />

students, with<br />

leaders <strong>on</strong> the<br />

side (4)<br />

led by<br />

students, with<br />

leaders hands-<br />

off (5)<br />

Rate the activities and discussi<strong>on</strong>s <strong>on</strong> a spectrum from:<br />

(1) Solely Social.....................................to......................................Serious Studies<br />

(5)<br />

C<strong>on</strong>structivism involves pers<strong>on</strong>al goals, knowledge c<strong>on</strong>structi<strong>on</strong> and interpretati<strong>on</strong>, and multiple<br />

perspectives <strong>on</strong> issues. C<strong>on</strong>structivist learning is characterised by active learning, independent<br />

71<br />

5.0<br />

3.5<br />

3.5


Ruth de Villiers and Marco Cobus Pretorius<br />

research, collaborati<strong>on</strong>, applicati<strong>on</strong> to authentic tasks, and real-world situated learning. Customisati<strong>on</strong><br />

entails learner-centricity and adaptability, allowing learners to take initiative regarding (some of) the<br />

c<strong>on</strong>tent, foci and circumstances of learning. Creativity is characterized by innovati<strong>on</strong> within<br />

functi<strong>on</strong>ality and by engagement and motivati<strong>on</strong> of learners.<br />

There was close c<strong>on</strong>sensus between the four evaluators’ ratings. Differences between ratings<br />

assigned to particular criteria never exceeded 1. Table 2 shows the average rating assigned to each<br />

criteri<strong>on</strong>, as well as the cross-criteri<strong>on</strong> average for each factor. In investigating the implementati<strong>on</strong> of<br />

c<strong>on</strong>structivism, evaluators acknowledged the social-c<strong>on</strong>structivist nature of interacti<strong>on</strong>s <strong>on</strong> the Forum.<br />

The nature of the Forum provided scope for participants’ pers<strong>on</strong>al insights and independent<br />

interpretati<strong>on</strong>s, and encouraged the applicati<strong>on</strong> of theoretical c<strong>on</strong>cepts to real-world phenomena<br />

bey<strong>on</strong>d the curriculum. The cross-criteri<strong>on</strong> average rating for c<strong>on</strong>structivism was 4.125. Customisati<strong>on</strong><br />

of learning was unanimously rated at 5.0, since participants could choose the time and place of their<br />

activities, while learner-centricity allowed them to initiate topics and match their needs by c<strong>on</strong>tributing<br />

(or not) in their preferred style. With regard to creativity, the expert evaluators’ cross-criteri<strong>on</strong> average<br />

was 4.625. They felt that Facebook provided a novel and engaging envir<strong>on</strong>ment for learning in a<br />

social c<strong>on</strong>text. It was supportive in that the learning occurred in an envir<strong>on</strong>ment that was attractive,<br />

friendly and familiar to most of the Group. The atmosphere fostered innovative strategies, such as<br />

posting links to academic articles and communicating <strong>on</strong>e-<strong>on</strong>-<strong>on</strong>e off-Forum.<br />

Ratings <strong>on</strong> the Forum’s social climate averaged 4.33. Evaluators regretted that participati<strong>on</strong> was not<br />

higher, but in their open-ended resp<strong>on</strong>ses, summarized this academic venture <strong>on</strong> Facebook as a<br />

‘very positive experience’ and a ‘novel way of using social media’, where ‘students got to know each<br />

other academically’. The expert evaluators recognised the community as a ‘platform of trust’ with<br />

‘positive energetic vibes’, and ‘enough moderati<strong>on</strong> to ensure correct feedback without dampening the<br />

student voice’. In evaluating management and facilitati<strong>on</strong>, two evaluators selected ‘3’, and two<br />

assigned ‘4’, thus averaging 3.5 and indicating a well-moderated Forum, yet primarily student-led.<br />

Similarly, <strong>on</strong> the spectrum between ‘solely social’ and ‘serious studies’, two chose ‘3’ and two chose<br />

‘4’, again averaging 3.5, right of centre and indicating sound balance but str<strong>on</strong>ger <strong>on</strong> the academic<br />

aspects.<br />

6. C<strong>on</strong>clusi<strong>on</strong>s<br />

This secti<strong>on</strong> summarizes the findings by c<strong>on</strong>cisely re-visiting the research questi<strong>on</strong> and by<br />

highlighting findings that c<strong>on</strong>tribute to new knowledge about ODF’s <strong>on</strong> social networks.<br />

Data collecti<strong>on</strong> and analysis involved data triangulati<strong>on</strong> and methodological triangulati<strong>on</strong>. The findings<br />

of two studies, namely, qualitative analysis of free-text data from students’ reflective essays and<br />

heuristic evaluati<strong>on</strong> by experts of c<strong>on</strong>tributi<strong>on</strong>s to the discussi<strong>on</strong> forum, c<strong>on</strong>firmed each other, and<br />

thus provided a positive answer to the research questi<strong>on</strong>:<br />

Did the venture serve both social- and study-related pursuits in a synergistic manner?<br />

Synergy results when the combinati<strong>on</strong> of factors produces a joint impact greater than the sum of their<br />

separate effects. This Facebook venture was indeed synergistic, as students benefitted both socially<br />

and academically in social-c<strong>on</strong>structivist interacti<strong>on</strong>. The social setting strengthened the academic<br />

interacti<strong>on</strong>s while, c<strong>on</strong>versely, academic discourse in the eLearning domain provided a b<strong>on</strong>d that<br />

related them socially as peers with similar interests.<br />

Free-text essays articulating the students’ own percepti<strong>on</strong>s were analysed qualitatively, and forum<br />

interacti<strong>on</strong>s were evaluated heuristically according to c<strong>on</strong>temporary learning paradigms with human<br />

values. Both sets of results indicated a harm<strong>on</strong>ious social climate that fostered meaningful academic<br />

discussi<strong>on</strong>s, as participants posted, resp<strong>on</strong>ded, received feedback and gained insights that enhanced<br />

their studies. C<strong>on</strong>versely, the study-related pursuits of research, interpretati<strong>on</strong>, and discussi<strong>on</strong>s <strong>on</strong><br />

theoretical c<strong>on</strong>cepts, led to social negotiati<strong>on</strong> and interpers<strong>on</strong>al c<strong>on</strong>necti<strong>on</strong>s. The supportive ethos<br />

encouraged most members to be forthcoming, while others, feeling inadequate, experienced the<br />

Group as a safe human envir<strong>on</strong>ment for learning without c<strong>on</strong>tributing.<br />

Moreover, the findings provide new informati<strong>on</strong> regarding the climate and culture that can be obtained<br />

in an ODF <strong>on</strong> Facebook. Some important points are summarized:<br />

72


Ruth de Villiers and Marco Cobus Pretorius<br />

The nature of discourse in the supportive Facebook envir<strong>on</strong>ment emulated face-to-face<br />

postgraduate c<strong>on</strong>tact, providing a percepti<strong>on</strong> that the distance-learners actually knew each other.<br />

Participants’ real-world pers<strong>on</strong>alities became evident as they exercised their individual<br />

communicati<strong>on</strong> and learning styles within the virtual community.<br />

There was an ethos of voluntary sharing, rather than a culture of academic competitiveness.<br />

The envir<strong>on</strong>ment was facilitated in a way that encouraged student-centricity, yet the forum was<br />

effectively moderated when necessary.<br />

This study showed a synergistic balance of social- and study-related aspects, c<strong>on</strong>ducive to studies<br />

and to social engagement, but with a str<strong>on</strong>ger focus <strong>on</strong> academia. Although not all the students in the<br />

cohort joined or c<strong>on</strong>tributed actively, formerly isolated distance-learners in the Group became a<br />

community of practice in the domain of eLearning. These findings should encourage academics to<br />

establish groups and discussi<strong>on</strong> forums <strong>on</strong> social networks.<br />

References<br />

Bosch, T.E. (2009). Using <strong>on</strong>line social networking for teaching and learning: Facebook use at the University of<br />

Cape Town. Communicatio 35(2), pp 185-200.<br />

Creswell J.W. (2009). Research Design: Qualitative, Quantitative, and Mixed Methods Approaches. SAGE<br />

Publicati<strong>on</strong>s Inc.<br />

de Villiers, M.R. (2010). <strong>Academic</strong> use of a Group <strong>on</strong> Facebook: Initial findings and percepti<strong>on</strong>s. Proceedings of<br />

the 2010 Informing Science and IT Educati<strong>on</strong> (InSITE) <str<strong>on</strong>g>C<strong>on</strong>ference</str<strong>on</strong>g>. Cassino, Italy, June 2010.<br />

Ebner, M. (2007). ELearning 2.0 = eLearning 1.0 + Web 2.0. Proceedings of the Sec<strong>on</strong>d Internati<strong>on</strong>al<br />

<str<strong>on</strong>g>C<strong>on</strong>ference</str<strong>on</strong>g> <strong>on</strong> Availability, Reliability and Security (ARES'07), pp 1235-1239.<br />

Hew, K.F. (2011). Students’ and teachers’ use of Facebook. Computers in Human Behavior 27, pp 662-676.<br />

J<strong>on</strong>es, N., Blackey, H., Fitzgibb<strong>on</strong>, K. and Chew, E. (2010). Get out of MySpace! Computers & Educati<strong>on</strong> 54, pp<br />

776-782.<br />

Lampe, C., Ellis<strong>on</strong>, B.N. and Steinfield, C. (2008). Changes in use and percepti<strong>on</strong> of Facebook. Proceedings of<br />

CSCW’08, pp 721-730, San Diego, California, USA.<br />

Madge, C. Meek, J., Wellens, J. and Hooley T. (2009). Facebook, social integrati<strong>on</strong> and informal learning at<br />

university: ‘It is more for socializing and talking to friends about work than for actually doing work’. Learning,<br />

Media and Technology 34(2), pp141-155.<br />

Mazman, S.G. and Usluel, Y.K. (2010). Modeling educati<strong>on</strong>al usage of Facebook. Computers & Educati<strong>on</strong> 55: pp<br />

444-453.<br />

Ractham, P. and Firpo, D. (2011). Using social networking technology to enhance learning in higher educati<strong>on</strong>: A<br />

case study using Facebook. Proceedings of 44th Hawaii Internati<strong>on</strong>al <str<strong>on</strong>g>C<strong>on</strong>ference</str<strong>on</strong>g> <strong>on</strong> System Sciences<br />

2011.<br />

Rambe, P. and Ng’ambi, D. (2011). Towards an Informati<strong>on</strong> Sharing Pedagogy: A Case of Using Facebook in a<br />

Large First-Year Class. Informing Science: The Internati<strong>on</strong>al Journal of an Emerging Transdiscipline, 14, pp<br />

61-89.<br />

Selwyn, N. (2009). Faceworking: exploring students’ educati<strong>on</strong>-related use of Facebook. Learning, Media and<br />

Technology 34(2), pp 157-174.<br />

Visagie, S. and de Villiers, C. (2010). The c<strong>on</strong>siderati<strong>on</strong> of Facebook as an academic tool by ICT lecturers across<br />

five countries. Proceedings of SACLA 2010, <str<strong>on</strong>g>C<strong>on</strong>ference</str<strong>on</strong>g> of South African Computer Lecturers’ Associati<strong>on</strong>.<br />

South Africa.<br />

73


Evaluating the Process of Delivering Compelling Value<br />

Propositi<strong>on</strong>s: The Case of Mobile Payments<br />

Denis Dennehy, Frederic Adam and Fergal Cart<strong>on</strong><br />

Business Informati<strong>on</strong> Systems Department, University College Cork, Ireland<br />

d.j.dennehy@umail.ucc.ie<br />

fadam@afis.ucc.ie<br />

f.cart<strong>on</strong>@ucc.ie<br />

Abstract: The provisi<strong>on</strong> of mobile ph<strong>on</strong>ed based payments (m-payments) services to the general public requires<br />

the cooperati<strong>on</strong> of a number of specific stakeholders, each c<strong>on</strong>tributing part of the overall soluti<strong>on</strong>, but also each<br />

with their different motives, resources and capabilities to deliver compelling value propositi<strong>on</strong>s to c<strong>on</strong>sumers. The<br />

need to combine these multiple perspectives makes exploring the requirements for the design and<br />

implementati<strong>on</strong> of m-payment initiatives a complex activity. As a result, sustainable ec<strong>on</strong>omic business models<br />

have yet to emerge for m-payment scenarios. The business model c<strong>on</strong>cept has been amply dem<strong>on</strong>strated to be<br />

a very useful tool to understand how to design commercially viable offerings over the last 15 years. The purpose<br />

of this research is to theorise the linkages between the numerous elements of an m-payment business model and<br />

to evaluate the process whereby stakeholders in an m-payments ecosystem create, exchange and deliver an mpayment<br />

soluti<strong>on</strong>. By leveraging the business model c<strong>on</strong>cept and the process modelling techniques associated<br />

with it, the research represents the various stakeholders, in an ec<strong>on</strong>omic value c<strong>on</strong>text, by identifying the role<br />

played by each stakeholder and the share of the profit which they expect in return for their c<strong>on</strong>tributi<strong>on</strong>. The study<br />

is unique in that it tracks the activities and decisi<strong>on</strong>s of stakeholders involved in a real-world m-payment initiative<br />

from c<strong>on</strong>cept stage through to launch stage. As a result, the study provides new insights into the complex and<br />

sensitive issues that need to be c<strong>on</strong>sidered by practiti<strong>on</strong>ers, while also providing researchers with a balanced<br />

and holistic perspective to a complex phenomen<strong>on</strong>. It also leverages the business model c<strong>on</strong>cept to present<br />

detailed models of the m-payment soluti<strong>on</strong> implemented in the case study. The preliminary data which we<br />

abstracted from the case has validated the relevance of the research questi<strong>on</strong>s and will be a valuable<br />

c<strong>on</strong>tributi<strong>on</strong> to the future creati<strong>on</strong> of an m-payment c<strong>on</strong>sortium.<br />

Keywords: mobile payments, M-payments, business model, business process, value networks, value webs<br />

1. Introducti<strong>on</strong><br />

Due to the fundamental attributes of mobile business (i.e. anytime, anywhere, <strong>on</strong> any device),<br />

organisati<strong>on</strong>s are increasingly leveraging such enabling technologies to create value to support<br />

mobile users (i.e. employees, c<strong>on</strong>sumers) or mobile activities (i.e. tracking materials or products), to<br />

improve their operati<strong>on</strong>s and subsequently to increase their competitive advantage and financial profit<br />

(Coursaris et al., 2006). For example, financial instituti<strong>on</strong>s are investing in new payment systems in<br />

order to reduce their operating costs, generate new revenue via new fees, and develop their customer<br />

relati<strong>on</strong>ship, early mover advantage (Dahlberg et al., 2008; FINsights, 2008).<br />

However, if organisati<strong>on</strong>s intend to exploit the attributes unique to mobile business, they will need to<br />

do so within the c<strong>on</strong>text of “effective business models that clearly articulate a compelling value<br />

propositi<strong>on</strong> for their employees and customers while addressing their various c<strong>on</strong>cerns” (Coursaris et<br />

al., 2006, p.7). To create value, organisati<strong>on</strong>s will need to learn from past business failures and<br />

develop sustainable business models using c<strong>on</strong>venti<strong>on</strong>al key performance indicators (KPI’s) and not<br />

rely <strong>on</strong> advertising and branding al<strong>on</strong>e (Cassidy, 2003).<br />

It is unclear why mobile services have not lived up to the hype or expectati<strong>on</strong>s as promised by its<br />

prop<strong>on</strong>ents (Damsgaard and Hedman, 2009), there are suggesti<strong>on</strong>s that the main issues are poor<br />

revenue sharing am<strong>on</strong>gst stakeholders (Ball<strong>on</strong> and Van Bossuyt, 2006) or static business models<br />

being used in the complex envir<strong>on</strong>ment which is required to deliver mobile services (Coursaris et al.,<br />

2006). The uncertainty around establishing a sustainable ec<strong>on</strong>omic business model that can be<br />

agreed by the multiple stakeholders has been identified as a c<strong>on</strong>tributing factor for the delayed launch<br />

of m-payment initiatives (Chaix and Torre, 2010). What has been established and is generally<br />

accepted by researchers and practiti<strong>on</strong>ers is that because the c<strong>on</strong>text of every m-payment service will<br />

be different, then every m-payment soluti<strong>on</strong> needs to be specifically customised to satisfy key actors<br />

in the m-payment ecosystem (Ondrus and Lyytinen, 2011; FINsights, 2008). These actors include:<br />

c<strong>on</strong>sumers, merchants, mobile network operators, mobile device manufacturers, financial instituti<strong>on</strong>s,<br />

software and technology providers, and governments (Dahlberg et al., 2007).<br />

74


Denis Dennehy, Frederic Adam and Fergal Cart<strong>on</strong><br />

Since no single actor can deliver an end-to-end m-payment service, the success of m-commerce<br />

relies <strong>on</strong> partnerships, thus making ‘partnership management’ a core competence that will enable<br />

stakeholders in the ecosystem to form viable alliances and actor networks (Camp<strong>on</strong>ovo and Pigneur,<br />

2002; Pigneur, 2002). Str<strong>on</strong>g partnerships and alliances not <strong>on</strong>ly lead to high levels of trust and<br />

cooperati<strong>on</strong> but they also enable organisati<strong>on</strong>s to achieve market leadership which in turn increases<br />

their market reach to co-opt c<strong>on</strong>sumers or suppliers within their value-network (Lewis et al., 2003;<br />

Currie and Parikh, 2006). However, partnerships and alliances are just <strong>on</strong>e strategic approach to<br />

enhance market leadership, too many partners without strategic market positi<strong>on</strong>ing could result in<br />

weak or unrealistic partnerships leading to business failure (Currie and Parikh, 2006).<br />

While building alliances is <strong>on</strong>e of the most important strategic approaches to creating value in ebusiness,<br />

it is not the <strong>on</strong>ly approach for c<strong>on</strong>tent and network providers to enter the market or increase<br />

their competitive advantage (Camp<strong>on</strong>ovo and Pigneur, 2003). In order to increase their competitive<br />

advantage, organisati<strong>on</strong>s will exploit their market positi<strong>on</strong>, negotiating power and access to critical<br />

resources (Bouwman and Ham, 2003). In additi<strong>on</strong>, access to key functi<strong>on</strong>s of billing and informati<strong>on</strong><br />

sharing is emerging as a critical success factor in the competiti<strong>on</strong> and development of sustainable<br />

business models (ibid). For example, in the c<strong>on</strong>text of payment services, financial instituti<strong>on</strong>s have a<br />

l<strong>on</strong>g traditi<strong>on</strong> of cooperating with merchants whereas cooperating with telecoms and technology<br />

vendors is a new experience for them (Dahlberg et al., 2008). Therefore, examining the actor’s roles<br />

is not sufficient, the relati<strong>on</strong>ships and interacti<strong>on</strong>s between the actors will also need to be assessed<br />

(Camp<strong>on</strong>ovo and Pigneur, 2002; Pigneur, 2002).<br />

Even though c<strong>on</strong>siderable research has already been c<strong>on</strong>ducted to better understand different<br />

aspects of the m-payment phenomen<strong>on</strong>, undertaking this study would answer the call for research<br />

that will lead to an ‘integrated view’ <strong>on</strong> m-payment business models (Pousttchi et al., 2009) as the<br />

success or failure of previous m-payment initiatives were based <strong>on</strong> issues arising out of multiple<br />

perspectives rather than a single perspective (Ondrus et al., 2005).<br />

2. Value-webs<br />

The value chain framework was initially intended to analyse traditi<strong>on</strong>al manufacturing industries <strong>on</strong>ly,<br />

but in recent times it has been used to analyse the mobile industry. However, the framework needs to<br />

c<strong>on</strong>sider other value c<strong>on</strong>figurati<strong>on</strong>s (i.e. value net) which better represent the mobile industry. Rather<br />

than <strong>on</strong>ly focusing <strong>on</strong> infrastructure and activities, a more comprehensive analysis <strong>on</strong> elements such<br />

as customer relati<strong>on</strong>ship, value propositi<strong>on</strong> and partnerships can be achieved by applying the<br />

business model c<strong>on</strong>cept (Camp<strong>on</strong>ovo and Pigneur, 2003).<br />

In a complex value network or value-web (i.e. m-payment ecosystem) where the organisati<strong>on</strong>s are<br />

engaged in inter-organisati<strong>on</strong>al investments, they are c<strong>on</strong>nected through intended relati<strong>on</strong>ships and<br />

interdependencies which involve c<strong>on</strong>siderable risks, problem solving and having access to<br />

complimentary knowledge (Dahlberg et al., 2008; Bouwman and Ham, 2003). This complexity and<br />

combinati<strong>on</strong> inevitably requires such organisati<strong>on</strong>s to undertake a collective-decisi<strong>on</strong> process<br />

(Bouwman and Ham, 2003). Unlike the traditi<strong>on</strong>al static and linear value-chain, value-webs are<br />

flexible and each stakeholder will possess different capabilities and resources which will lead to<br />

innovative soluti<strong>on</strong>s, when combined together (Moschella, 2003; Faber et al., 2004). As organisati<strong>on</strong>s<br />

shift from single firm revenue generati<strong>on</strong> to multi-firm c<strong>on</strong>trol and revenue sharing issues, not <strong>on</strong>ly are<br />

c<strong>on</strong>trol and value issues of most relevance to business modelling, but two key questi<strong>on</strong>s also emerge<br />

“ Who c<strong>on</strong>trols the value network and the system design?” and “ Is substantial value being produced<br />

by this model or not?” (Ball<strong>on</strong>, 2007, p. 2).<br />

Nevertheless, these collective-decisi<strong>on</strong> processes have a number of implicati<strong>on</strong>s when compared to<br />

internal process since no single partner has formal authority over another partner; they require<br />

prol<strong>on</strong>ged decisi<strong>on</strong>-making processes as adjustments need to be discussed and jointly agreed, they<br />

demand several rounds of negotiati<strong>on</strong>s, there are high costs involved and the possibility of disputes<br />

due to c<strong>on</strong>flicting interests which do not always result in a win-win outcome for all stakeholders<br />

(Demkes, 1999; Klein-Woolthuis et al., 2005; Faber et al., 2004). There are three types of participants<br />

in any new value network (see Table 1): at the core of the network are the structural partners while<br />

the c<strong>on</strong>tributing and support partners are loosely linked to the network (Bouwman and Ham, 2003).<br />

75


Denis Dennehy, Frederic Adam and Fergal Cart<strong>on</strong><br />

Table 1: Partnership tier (adapted from Bouwman and Ham, 2003)<br />

Level Role<br />

Tier 1 (Structural) Partners provide essential and n<strong>on</strong>-substitutable tangible and/or intangible<br />

assets to the enterprise <strong>on</strong> an equity or n<strong>on</strong>-equity basis. They play a direct and<br />

core role in making the customer value assumpti<strong>on</strong> and in creating the business<br />

model.<br />

Tier 2 (C<strong>on</strong>tributing) Partners provide goods and/or services to meet requirements that are specific to<br />

the enterprise, but otherwise they play no direct role in making the customer<br />

value assumpti<strong>on</strong> and in creating the business model. If the assets they provide<br />

are substituted, the value assumpti<strong>on</strong> and the business model could still stand.<br />

Tier 3 (Support) Partners provide generic goods and services to the enterprise, without which the<br />

enterprise would not be viable, but which otherwise could be used in c<strong>on</strong>necti<strong>on</strong><br />

with a wide variety of value assumpti<strong>on</strong>s and business models.<br />

Table 1 is significant as it indicates that not <strong>on</strong>ly do all partners have a role in a value-creating<br />

network, whether its playing an important role and having influence in shaping the network or having a<br />

minor role and being shaped by the network, such a network inevitably requires ‘partnership<br />

management’ (Galbreath, 2002; Kothandaraman and Wils<strong>on</strong>, 2001). Adopting the partnership tier<br />

may be beneficial in order to overcome the symptoms of negative dynamics between some actors<br />

which has resulted in “misjudged resource strengths, complementary assets, and market size<br />

evaluati<strong>on</strong>s” in previous m-payment initiatives (Dahlberg et al., 2008, p. 9).<br />

Cooperati<strong>on</strong> in a value-web is challenging as there is evidence that the organisati<strong>on</strong>s experience<br />

significant difficulties in attaining mutual benefits from co-operati<strong>on</strong> because each partner may be<br />

pursuing strategic goals that are different to the co-operati<strong>on</strong> which in turn may lead to hiding of the<br />

truth or acquiring sensitive informati<strong>on</strong> from the partners, because the partners are from different<br />

industries (i.e. retailers, network providers) there is potential that the diversity could disrupt the<br />

ecosystem, and revenue sharing issues (Faber et al., 2004; Ball<strong>on</strong>, 2007). These challenges can be<br />

partly attributed to explain why the success of m-payment platforms remain hampered by recurring<br />

and fundamental social, instituti<strong>on</strong>al and business challenges that requires a multi-level and multiperspective<br />

holistic approach as it provides a richer picture of the phenomen<strong>on</strong> (Gao and Damsgaard,<br />

2007; Ondrus and Lyytinen, 2011; Currie and Parikh, 2006; Dahlberg et al., 2008). This means that<br />

researching m-payment adopti<strong>on</strong> issues without assessing the instituti<strong>on</strong>al and business c<strong>on</strong>text will<br />

not provide sufficient explanati<strong>on</strong>s into a complex, networked technology such as m-payments<br />

(Zmijewska and Lawrence, 2005; Ondrus and Lyytinen, 2011).<br />

3. Leveraging the business model c<strong>on</strong>cept<br />

<strong>Academic</strong>s have increasingly given attenti<strong>on</strong> to developing the business model c<strong>on</strong>cept by defining<br />

business models, examining their comp<strong>on</strong>ents, classifying them into categories and more recently<br />

focusing <strong>on</strong> representati<strong>on</strong>s or developing descriptive models. Yet, there is a paucity of research that<br />

theorises the linkages between the variables of the numerous business model domains (e.g. service,<br />

finance, organisati<strong>on</strong>, technology), and more specifically, business models used by networked<br />

organisati<strong>on</strong>s (Faber et al., 2004; Bouwman and Ham, 2003).<br />

When assessing the role of different stakeholders in the m-commerce ecosystem it is suggested to<br />

briefly and clearly describe their business models (Camp<strong>on</strong>ovo and Pigneur, 2002). Business models<br />

also offer a high level of abstracti<strong>on</strong> which is the correct starting point when creating or redefining<br />

business processes rather than analysing the business processes themselves (Weigand et al., 2006).<br />

Business models and process models provide different support for decisi<strong>on</strong>s and requirements of<br />

different stakeholders. By commencing a project with a business model design, it states “what is<br />

offered by whom to whom, rather than how these value-creating activities are selected, negotiated,<br />

c<strong>on</strong>tracted and fulfilled operati<strong>on</strong>ally” as is explained by a process model (Gordijn et al., 2000, p. 1).<br />

A business model represents the interplay between multiple industries (Chesbrough and Appleyard,<br />

2007) and is beneficial in determining the underlying logic that explains how an organisati<strong>on</strong> creates<br />

and delivers value to their customers while also capturing returns from that value (Magretta, 2002;<br />

Shafer et al., 2005). A business model can be an influential tool for analysing, implementing and<br />

communicating strategic choices (Magretta, 2002; Shafer et al., 2005). As the functi<strong>on</strong>s of a business<br />

model are to articulate the value propositi<strong>on</strong>, identify a market segment, define the structure of the<br />

value chain and to estimate the cost structure and profit potential (Chesbrough and Rosenbloom,<br />

2002), they are an important locus of innovati<strong>on</strong> and are a critical source of value creati<strong>on</strong> and<br />

76


Denis Dennehy, Frederic Adam and Fergal Cart<strong>on</strong><br />

competitive advantage for an organisati<strong>on</strong>, its suppliers, partners, and customers (Amit and Zott,<br />

2001; Mitchell and Coles, 2003).<br />

Although the business model c<strong>on</strong>cept has been criticised for its “murky definiti<strong>on</strong>s and… loose<br />

c<strong>on</strong>cepti<strong>on</strong> of how a company does business and generates revenue” (Porter, 2001, p. 13), the<br />

business model c<strong>on</strong>cept can be strengthened by aligning it with established theories (e.g. innovati<strong>on</strong><br />

management, strategic management, resource-based theory) that also deal with c<strong>on</strong>trol and value<br />

creati<strong>on</strong> issue in a network (Ball<strong>on</strong>, 2007). Nevertheless, business models have an important role in<br />

business practice (Bodker et al., 2009) because a good business model is essential for any<br />

organisati<strong>on</strong> as it answers fundamental questi<strong>on</strong>s such as “Who is the customer?” and “What does<br />

the customer value” (Magretta, 2002), and it is a vital source of value creati<strong>on</strong> for an organisati<strong>on</strong>s<br />

suppliers, partners and customers, as well as for innovati<strong>on</strong> (Amit and Zott, 2001).<br />

In the c<strong>on</strong>text of explaining mobile business there is extensive knowledge available <strong>on</strong> descriptive<br />

business models and value systems, but there is a lack of models (i.e. causal models) which explain<br />

the viability and feasibility of business models, as are case-related analysis and cross-secti<strong>on</strong>al data<br />

(Bouwman and Ham, 2003). Aligning the business model with the market envir<strong>on</strong>ment as well as the<br />

industry envir<strong>on</strong>ment is crucial (Ondrus et al., 2009). Yet, a majority of managers find the business<br />

model c<strong>on</strong>cept difficult as they either d<strong>on</strong>’t understand their current model enough when to know it<br />

needs changing or how to make that change (Chesbrough and Schwartz, 2007; Johns<strong>on</strong> et al.,<br />

2008).. Business models in a multi-sided network are about getting stakeholders <strong>on</strong> board, balancing<br />

their respective interests and locking c<strong>on</strong>sumers in single or multiple platforms (i.e. multi-homing).<br />

(Ball<strong>on</strong> et al., 2008). When managers c<strong>on</strong>sciously operate from a model of how the entire business<br />

systems will work, every decisi<strong>on</strong>, initiative, and measurement provides valuable feedback (Magretta,<br />

2002).<br />

Business model innovati<strong>on</strong> is the discovery of a fundamentally different business model in an existing<br />

business, business model innovators redefine an existing product or service and how it is delivered to<br />

the customer (Markides, 2006). Although the ultimate aim of designing a business model is to create<br />

sufficient ec<strong>on</strong>omic and customer value, the challenge is that it requires c<strong>on</strong>necting and balancing<br />

design choices in different business model domains while taking into account technical, market and<br />

legal developments (Faber et al., 2004). In many instances, the customer value envisaged in the<br />

initial design of a business model has little to do with the value as perceived by the customer (e.g. the<br />

end-user) as it depends <strong>on</strong> the customers pers<strong>on</strong>al and c<strong>on</strong>sumpti<strong>on</strong> c<strong>on</strong>text (Chen and Dubinsky,<br />

2003; Wieringa and Gordijn, 2005). By understanding the critical design issues in business models<br />

and their interdependencies, rather than identifying relatively easy prescripti<strong>on</strong>s, new insights are<br />

inserted into the design of ‘balanced’ business models as knowledge of how to effectively balance<br />

requirements and strategic interests is limited in the business model literature (Faber et al., 2004).<br />

Adopting the Osterwalder et al (2005) business model c<strong>on</strong>cept and aligning the research questi<strong>on</strong>s to<br />

each pillar, (see Table 2), the authors believe that a multi-level and multi-perspective understanding<br />

into the design and delivery of an m-payment initiative. As such, the business model c<strong>on</strong>cept can be<br />

used as a vehicle for innovati<strong>on</strong> and also as a subject of innovati<strong>on</strong> (Zott et al., 2011). From a<br />

practical perspective, the Osterwalder et al (2005) business model provides very str<strong>on</strong>g support to<br />

this research project as it proposes a complete set of elements, relati<strong>on</strong>ships and vocabulary to<br />

describe and analyse a business model (Pousttchi et al., 2009).<br />

Table 2: Research questi<strong>on</strong>s within the business model c<strong>on</strong>cept<br />

Research Questi<strong>on</strong> Pillar Building Block of Business<br />

RQ 1: How does the adopti<strong>on</strong> of an NFC system change the<br />

value propositi<strong>on</strong>s offered by service providers?<br />

Product Value Propositi<strong>on</strong><br />

RQ 2: How does the adopti<strong>on</strong> of an NFC system change the Customer Target Customer<br />

customer interface?<br />

Interface Distributi<strong>on</strong> Channel<br />

RQ 3: How does the adopti<strong>on</strong> of NFC systems change the<br />

infrastructure management of the m-payments network?<br />

RQ 4: How can m-payment service providers agree to share<br />

the costs and revenues associated with the delivery of an mpayment?<br />

77<br />

Relati<strong>on</strong>ship<br />

Value C<strong>on</strong>figurati<strong>on</strong><br />

Infrastructure<br />

<strong>Management</strong> Core Competency<br />

Partner Network<br />

Financial<br />

Cost Structure<br />

Aspects Revenue Model


Denis Dennehy, Frederic Adam and Fergal Cart<strong>on</strong><br />

The research questi<strong>on</strong>s are designed enable the researchers to examine the effect of NFC enabled<br />

payment systems <strong>on</strong> business model development. We are applying these questi<strong>on</strong>s to a case study<br />

of the development and executi<strong>on</strong> of an m-payment trial <strong>on</strong> a university campus. The method<br />

employed by the researchers to investigate the research questi<strong>on</strong>s are presented in the next secti<strong>on</strong>.<br />

4. Case study<br />

The data required to achieve the research objective and the associated research questi<strong>on</strong>s will be<br />

acquired from a real-world m-payment project which will take place <strong>on</strong> the campus of University<br />

College Cork (UCC) and with 250 students as participants. The pilot m-payment project is being<br />

facilitated by the Financial Services Innovati<strong>on</strong> Centre (FSIC) in UCC and in collaborati<strong>on</strong> with a<br />

leading mobile ph<strong>on</strong>e network provider and other stakeholders (i.e. integrati<strong>on</strong> partners) from the<br />

retail and payments industries. The integrati<strong>on</strong> partners in the project include: MNO, handset and<br />

operating system manufacturer, the SIM card manufacturer and SIM card integrati<strong>on</strong> team, the mobile<br />

wallet applicati<strong>on</strong> developers, the funding account and card issuer, the payment transacti<strong>on</strong><br />

processor, the NFC terminal provider, and IT technicians from the host university.<br />

The data will be generated in the form of focus group interviews with the participants and face-to-face<br />

interviews with the retailers and the other stakeholders. The project will go-live in mid-2012 and<br />

interviews will be carried out at the pre-launch, mid-launch, and late-launch phases of the project. Todate<br />

preliminary data has been gathered and the findings have been useful in guiding the overall<br />

directi<strong>on</strong> of the pilot project and to validate the relevance of the research objective and research<br />

questi<strong>on</strong>s.<br />

5. Research method<br />

Adopting an exploratory research approach is appropriate for this study as it is particularly useful in<br />

highly innovative industries, as well as to develop a better understanding of the business problem (i.e.<br />

how to deliver compelling value propositi<strong>on</strong>s to c<strong>on</strong>sumers) by discovering new relati<strong>on</strong>ships or<br />

patterns (Hair et al., 2007). Further, the use of case studies permits the researcher to examine the<br />

phenomen<strong>on</strong> in its natural setting by employing multiple methods of data collecti<strong>on</strong> to gather<br />

informati<strong>on</strong> from the different stakeholders with the goal of acquiring a rich set of data (Benbasat et<br />

al., 1987; Denzin and Lincoln, 2000).<br />

Case studies are also suitable for researching an area in which there is paucity of research and to<br />

finding answers to “how” and “why” questi<strong>on</strong>s (Franz and Robey, 1984; Benbasat et al., 1987). The<br />

benefit of employing a variety of data collecti<strong>on</strong> techniques is that collectively they offset limitati<strong>on</strong>s<br />

specific to each individual technique while also addressing possible anomalies (Gallivan, 1997;<br />

McGrath, 1984) and providing the opportunity to triangulate findings, thus reinforcing the c<strong>on</strong>clusi<strong>on</strong>s<br />

of the study (Kelliher, 2005; Benbasat et al., 1987).<br />

The study will use focus groups and face-to-face interviews to provide a rich set of data, as well as<br />

capturing the c<strong>on</strong>textual complexity of m-payments (Yin, 1984; Benbasat et al., 1987; Remenyi and<br />

Williams, 1995). Focus group interviews are suitable when the goal of using qualitative research is to<br />

generate theoretical ideas and hypotheses which they intend to verify with future quantitative research<br />

(Calder, 1977) and have been employed in previous mobile service studies (Jarvenpaa and Lang,<br />

2005; Garfield, 2005). Participants will be selected from naturally formed groups (i.e. college class) as<br />

such groups tend be more relaxed and at ease in c<strong>on</strong>versati<strong>on</strong>s (Bryman, 2001). Focus group<br />

interviews will also enable the researcher to gather large amounts of data quickly and provide the<br />

researcher with multiple perspectives (Wilkins<strong>on</strong>, 2004). Focus groups are more “naturalistic”, when<br />

compared with face-to-face interviews as they generally include a variety of communicative processes<br />

such as storytelling, disagreement, humour and cajoling am<strong>on</strong>gst the participants (ibid). Due to the<br />

sensitive commercial aspects of this pilot project, face-to-face interviews will also be employed in<br />

order to reveal issues specific to the various commercial stakeholders (i.e. retailers, mobile network<br />

operator (MNO), and financial acquirer) involved in the project.<br />

6. Preliminary findings<br />

The university campus would appear to be an ideal envir<strong>on</strong>ment to launch an m-payment service due<br />

to the ‘village’ effect; technology savvy c<strong>on</strong>sumers (i.e. BIS students and staff), merchants in the retail<br />

and restaurant business with over thirty point-of-sales (POS), a wireless campus envir<strong>on</strong>ment, and a<br />

host actor (FSIC) all within close proximity to each other.<br />

78


Denis Dennehy, Frederic Adam and Fergal Cart<strong>on</strong><br />

Nevertheless, a number of unanticipated challenges emerged which subsequently delayed the launch<br />

date for the project twice. While technological issues have been identified as fundamental reas<strong>on</strong> for<br />

the delay, other issues that threatened the roll-out of the NFC system included managing the partners<br />

in the value network, and the issue of project costs and transacti<strong>on</strong> costs remained “the elephant in<br />

the room”. For example, interviews with the merchants revealed that they anticipate the potential of<br />

m-payments in terms of it reducing the time and cost required to managing physical cash, as well as,<br />

a faster through-put at peak service times (e.g. 12pm - 2pm). However, merchants also expressed a<br />

high level of commitment to the project <strong>on</strong> c<strong>on</strong>diti<strong>on</strong> that they did not incur the cost of installing the<br />

NFC enabled terminals at the thirty three point-of-sales located <strong>on</strong> campus. Yet, even at the<br />

prelaunch phase, cost issues remained “unelaborated”. These issues are specific to research<br />

questi<strong>on</strong> three and four respectively: “How does the adopti<strong>on</strong> of NFC systems change the<br />

infrastructure management of the m-payments network?” and “How can m-payment service providers<br />

agree to share the costs and revenues associated with the delivery of an m-payment?” Specific to<br />

research questi<strong>on</strong> three, the researchers have identified that infrastructure management is a central<br />

issue that requires diplomacy, coordinati<strong>on</strong> and the need for a shared terminology am<strong>on</strong>gst the<br />

integrati<strong>on</strong> partners.<br />

Key to the adopti<strong>on</strong> by merchants of this new payment channel are the associated interchange costs.<br />

Negotiati<strong>on</strong>s between the payment processor and the merchants have been characterised by an<br />

extremely stilted discussi<strong>on</strong> which highlights the lack of experience in micro-payments am<strong>on</strong>g the<br />

acquirers of the transacti<strong>on</strong>s. Average values per transacti<strong>on</strong> at a campus POS are between €2 and<br />

€3. Certain known value c<strong>on</strong>fecti<strong>on</strong>ery items are sold at minimal cost compared to high street prices,<br />

with margins of 5-8%. When the payment processors c<strong>on</strong>sider applying standard acquisiti<strong>on</strong> fees to<br />

this level of purchase (for example, 14 cent per transacti<strong>on</strong>), it can be seen that merchant margin is<br />

wiped out. Furthermore, the merchant must open a merchant ID account with the acquirer, for a <strong>on</strong>e<br />

off fee of €250, and additi<strong>on</strong>ally spend €7 per terminal per year for support.<br />

From the acquirer’s point of view, merchant support and transacti<strong>on</strong> fees are based <strong>on</strong> traditi<strong>on</strong>al<br />

debit and credit card interchange rates available from the card networks. However, from the merchant<br />

viewpoint, the motivati<strong>on</strong> to promote a new payment channel needs will not come from eating into the<br />

tight margins <strong>on</strong> sub €15 spend items. On the c<strong>on</strong>trary, both acquirers and card networks will need to<br />

rec<strong>on</strong>sider the value propositi<strong>on</strong> to merchants (and c<strong>on</strong>sumers) if critical mass is to be reached <strong>on</strong> mpayment<br />

adopti<strong>on</strong>. These findings present an early indicati<strong>on</strong> that the cultural barriers to new<br />

business model development are significant, with players inevitably burdened by their inherited<br />

percepti<strong>on</strong>s of customer value propositi<strong>on</strong>s.<br />

Such issues dem<strong>on</strong>strate the need to identify how the stakeholders perceive themselves and the<br />

other stakeholders within the value network and in the c<strong>on</strong>text of a partnership tier framework. The<br />

absence of such a framework may partly explain why there was no explicit “lead actor” in the project<br />

or why the stakeholders, who are established players in their own industries, engaged in a project<br />

without having a clear business model from the out-set to ensure a win-win outcome for all<br />

stakeholders.<br />

From the c<strong>on</strong>sumers perspective, the authors have gathered additi<strong>on</strong>al data by carrying out a number<br />

of focus group interviews at the pre-launch stage where participants (i.e. c<strong>on</strong>sumers) were invited to<br />

discuss the value-propositi<strong>on</strong>s been offered by the use of the NFC enabled ph<strong>on</strong>e and the service<br />

providers (e.g. research questi<strong>on</strong> 1). Key themes that were discussed included: their understanding of<br />

the m-payment c<strong>on</strong>cept, the value propositi<strong>on</strong>s that would entice them to a) migrate and b) remain<br />

with a new MNO, and the barriers that would prohibit them from participating in the trial. Participants<br />

str<strong>on</strong>gly favoured customer loyalty schemes that offered a range of opti<strong>on</strong>s, instant or short-term<br />

rewards, and the ability to use the rewards with other branded goods and services. Key barriers to<br />

participating in the project included: top-up charges, the use of a low-end smartph<strong>on</strong>e due to its<br />

limited functi<strong>on</strong>ality and the fear of technical errors at the POS.<br />

Unfortunately, the project was offically terminated by the MNO in May this year as the testing phase<br />

highlighted a number of “customer experience issues” that could not be addressed in time for a third<br />

launch attempt. Nevertheless, the MNO h<strong>on</strong>oured its commitment to provide the registered<br />

participants with a smartph<strong>on</strong>e, as agreed from the out-set. The data gathered at the pre-launch<br />

phase suggests that their is need for the creati<strong>on</strong> of a comm<strong>on</strong> language (i.e. <strong>on</strong>tology) and the<br />

development of a visualisati<strong>on</strong> tool in order to support multiple stakeholders to address key<br />

79


Denis Dennehy, Frederic Adam and Fergal Cart<strong>on</strong><br />

stakeholder issues at the early stage of forming a value-network and in c<strong>on</strong>juncti<strong>on</strong> with the<br />

envisaged business model. This new insight has prompted the authors to refine thier face-to-face<br />

interview questi<strong>on</strong>s with the stakeholders in order to acquire a retrospective and multi-perspective<br />

understanding into the complex m-payments phen<strong>on</strong>enom.<br />

References<br />

Amit, R. and Zott C. (2001). Value creati<strong>on</strong> in E business. Strategic <strong>Management</strong> Journal 22: 493-520.<br />

Ball<strong>on</strong>, P. (2007). Business modelling revisited: the c<strong>on</strong>figurati<strong>on</strong> of c<strong>on</strong>trol and value. info 9: 6-19.<br />

Ball<strong>on</strong>, P. and Van Bossuyt M. (2006). Comparing business models for multimedia c<strong>on</strong>tent distributi<strong>on</strong> platforms.<br />

IEEE, 15-15.<br />

Ball<strong>on</strong>, P., Walravens N., Spedalieri A. and Venezia C. (2008). The rec<strong>on</strong>figurati<strong>on</strong> of mobile service provisi<strong>on</strong>:<br />

towards platform business models.<br />

Benbasat, I., Goldstein D.K. and Mead M. (1987). The Case Research Strategy in Studies of Informati<strong>on</strong><br />

Systems. Mis Quarterly 11: 369-386.<br />

Bodker, M., Gimpel G. and Hedman J. (2009). Smart Ph<strong>on</strong>es and Their Substitutes: Task-Medium Fit and<br />

Business Models. Mobile Business, 2009. ICMB 2009. Eighth Internati<strong>on</strong>al <str<strong>on</strong>g>C<strong>on</strong>ference</str<strong>on</strong>g> <strong>on</strong>. 24-29.<br />

Bouwman, H. and Ham E. (2003). Designing metrics for business models describing Mobile services delivered by<br />

networked organisati<strong>on</strong>s. Citeseer.<br />

Bryman, A. (2001). Social science research methods. Oxford: Oxford University Press.<br />

Calder, B.J. (1977). Focus groups and the nature of qualitative marketing research. Journal of Marketing<br />

Research: 353-364.<br />

Camp<strong>on</strong>ovo, G. and Pigneur Y. (2002). Analyzing the actor game in m-business.<br />

Camp<strong>on</strong>ovo, G. and Pigneur Y. (2003). Analyzing the m-business landscape. Annals of telecommunicati<strong>on</strong>s 58:<br />

59-77.<br />

Cassidy, J. (2003). Dot. c<strong>on</strong>: how America lost its mind and m<strong>on</strong>ey in the Internet era: Harper Perennial.<br />

Chaix, L. and Torre D. (2010). Different models for mobile payments.<br />

Chen, Z. and Dubinsky A.J. (2003). A c<strong>on</strong>ceptual model of perceived customer value in e commerce: A<br />

preliminary investigati<strong>on</strong>. Psychology and Marketing 20: 323-347.<br />

Chesbrough, H. and Rosenbloom R.S. (2002). The role of the business model in capturing value from innovati<strong>on</strong>:<br />

evidence from Xerox Corporati<strong>on</strong>'s technology spin-off companies. Industrial and corporate change 11: 529.<br />

Chesbrough, H. and Schwartz K. (2007). Innovating business models with co-development partnerships.<br />

Research-Technology <strong>Management</strong> 50: 55-59.<br />

Chesbrough, H.W. and Appleyard M.M. (2007). Open innovati<strong>on</strong> and strategy. California <strong>Management</strong> Review<br />

50: 57.<br />

Coursaris, C., Hassanein K. and Head M. (2006). Mobile Technologies and the Value Chain: Participants,<br />

Activities and Value Creati<strong>on</strong>. Mobile Business, 2006. ICMB '06. Internati<strong>on</strong>al <str<strong>on</strong>g>C<strong>on</strong>ference</str<strong>on</strong>g> <strong>on</strong>. 8-8.<br />

Currie, W.L. and Parikh M.A. (2006). Value creati<strong>on</strong> in web services: an integrative model. The Journal of<br />

Strategic Informati<strong>on</strong> Systems 15: 153-174.<br />

Dahlberg, T., Huurros M. and Ainamo A. (2008). Lost Opportunity Why Has Dominant Design Failed to Emerge<br />

for the Mobile Payment Services Market in Finland? : IEEE, 83-83.<br />

Dahlberg, T., Mallat N., Ondrus J. and Zmijewska A. (2007). Past, present and future of mobile payments<br />

research: A literature review. Electr<strong>on</strong>ic Commerce Research and Applicati<strong>on</strong>s 7: 165-181.<br />

Damsgaard, J. and Hedman J. (2009). Mobile Services Revisited.<br />

Demkes, R. (1999). Comet. A Comprehensive Methodology for Supporting Telematics Investment Decisi<strong>on</strong>s.<br />

Denzin, N.K. and Lincoln Y.S. (2000). The discipline and practice of qualitative research. Handbook of qualitative<br />

research 2: 1-28.<br />

Faber, E., Haaker T. and Bouwman H. (2004). Balancing requirements for customer value of mobile services. 21-<br />

23.<br />

FINsights. (2008). Enterprise Payments. Technology Insights for the Financial Services Industry.<br />

Franz, C.R. and Robey D. (1984). An investigati<strong>on</strong> of user-led system design: rati<strong>on</strong>al and political perspectives.<br />

Communicati<strong>on</strong>s of the Acm 27: 1202-1209.<br />

Gallivan, M. (1997). Value in triangulati<strong>on</strong>: a comparis<strong>on</strong> of two approaches for combining qualitative and<br />

quantitative methods. Springer, 417.<br />

Gao, P. and Damsgaard J. (2007). A framework for understanding mobile telecommunicati<strong>on</strong>s market innovati<strong>on</strong>:<br />

A case of China. Journal of Electr<strong>on</strong>ic Commerce Research 8: 184-195.<br />

Garfield, M.J. (2005). Acceptance of ubiquitous computing. Informati<strong>on</strong> Systems <strong>Management</strong> 22: 24-31.<br />

Gordijn, J., Akkermans H. and Van Vliet H. (2000). Business Modelling is not Process Modelling. In C<strong>on</strong>ceptual<br />

Modeling for E-Business and the Web, ECOMO 2000. Springer.<br />

Hair, J.F., M<strong>on</strong>ey A.H., Samouel P. and Page M. (2007). Research methods for business: John Wiley & S<strong>on</strong>s.<br />

Hedman, J. and Kalling T. (2003). The business model c<strong>on</strong>cept: theoretical underpinnings and empirical<br />

illustrati<strong>on</strong>s. <str<strong>on</strong>g>European</str<strong>on</strong>g> Journal of Informati<strong>on</strong> Systems 12: 49-59.<br />

Jarvenpaa, S.L. and Lang K.R. (2005). Managing the paradoxes of mobile technology. Informati<strong>on</strong> Systems<br />

<strong>Management</strong> 22: 7-23.<br />

Johns<strong>on</strong>, M.W., Christensen C.M. and Kagermann H. (2008). Reinventing Your Business Model.<br />

Kelliher, F. (2005). Interpretivism and the pursuit of research legitimisati<strong>on</strong>: an integrated approach to single case<br />

design. Electr<strong>on</strong>ic Journal of Business Research Methods 3: 123-132.<br />

80


Denis Dennehy, Frederic Adam and Fergal Cart<strong>on</strong><br />

Klein-Woolthuis, R., Hillebrand B. and Nooteboom B. (2005). Trust, c<strong>on</strong>tract and relati<strong>on</strong>ship development.<br />

Organizati<strong>on</strong> Studies 26: 813-840.<br />

Lewis, W., Agarwal R. and Sambamurthy V. (2003). Sources of Influence <strong>on</strong> Beliefs about Informati<strong>on</strong><br />

Technology Use: An Empirical Study of Knowledge Workers. Mis Quarterly 27: 657-678.<br />

Magretta, J. (2002). Why business models matter. Harvard Business Review 80: 86-93.<br />

Markides, C. (2006). Disruptive Innovati<strong>on</strong>: In Need of Better Theory*. Journal of Product Innovati<strong>on</strong><br />

<strong>Management</strong> 23: 19-25.<br />

McGrath, J.E. (1984). Groups: Interacti<strong>on</strong> and performance: Prentice-Hall Englewood Cliffs, NJ.<br />

Mitchell, D. and Coles C. (2003). The ultimate competitive advantage of c<strong>on</strong>tinuing business model innovati<strong>on</strong>.<br />

Journal of Business Strategy 24: 15-21.<br />

Moschella, D. (2003). Customer-driven IT. Harvard Business School Press, Bost<strong>on</strong>.<br />

Ondrus, J. and Lyytinen K. (2011). Mobile Payments Market: Towards Another Clash of the Titans? Mobile<br />

Business (ICMB), 2011 Tenth Internati<strong>on</strong>al <str<strong>on</strong>g>C<strong>on</strong>ference</str<strong>on</strong>g> <strong>on</strong>. 166-172.<br />

Ondrus, J., Lyytinen K. and Pigneur Y. (2009). Why mobile payments fail? Towards a dynamic and multiperspective<br />

explanati<strong>on</strong>. IEEE, 1-10.<br />

Pigneur, Y. (2002). An Ontology for m-business models. C<strong>on</strong>ceptual Modeling—ER 2002: 3-6.<br />

Porter, M.E. (2001). Strategy and the Internet. Harvard Business Review 79: 62-79.<br />

Pousttchi, K., Schiessler M. and Wiedemann D.G. (2009). Proposing a comprehensive framework for analysis<br />

and engineering of mobile payment business models. Informati<strong>on</strong> Systems and E-Business <strong>Management</strong> 7:<br />

363-393.<br />

Remenyi, D. and Williams B. (1995). Some aspects of methodology for research in informati<strong>on</strong> systems. Journal<br />

of Informati<strong>on</strong> Technology 10: 191-201.<br />

Sedd<strong>on</strong>, P.B. and Lewis G.P. (2003). Strategy and business models: What's the difference. Citeseer.<br />

Shafer, S.M., Smith H.J. and Linder J.C. (2005). The power of business models. Business horiz<strong>on</strong>s 48: 199-207.<br />

Weigand, H., Johanness<strong>on</strong> P., Anderss<strong>on</strong> B., Bergholtz M., Edirisuriya A. and Ilayperuma T. (2006). On the<br />

noti<strong>on</strong> of value object. Springer, 321-335.<br />

Wieringa, R.J. and Gordijn J. (2005). Value-oriented design of service coordinati<strong>on</strong> processes: correctness and<br />

trust. ACM, 1320-1327.<br />

Wilkins<strong>on</strong>, S. (2004). Focus Group Research. Qualitative research: Theory, method and practice: 177.<br />

Yin, R.K. (1984). Case study research: design and methods. 2003. Applied Social Research Methods Series 5.<br />

Zmijewska, A. and Lawrence E. (2005). Reshaping the framework for analysing success of mobile payment<br />

soluti<strong>on</strong>s.<br />

Zott, C., Amit R. and Massa L. (2011). The Business Model: Recent Developments and Future Research. Journal<br />

of <strong>Management</strong> 37: 1019.<br />

81


Using Bricolage to Facilitate Emergent Collectives in SMEs<br />

Jan Devos 1 , Hendrik Van Landeghem 2 and Dirk Deschoolmeester 3<br />

1<br />

Ghent University, Faculty of Engineering and Architecture, Campus Kortrijk,<br />

Belgium<br />

2<br />

Ghent University, Faculty of Engineering and Architecture, Department of<br />

Industrial <strong>Management</strong>, Gent, Belgium<br />

3<br />

Ghent University, Faculty of Ec<strong>on</strong>omy and Business Administrati<strong>on</strong>, Gent<br />

Belgium<br />

jan.devos@howest.be,<br />

hendrik.vanlandeghem@ugent.be<br />

dirk.deschoolmeester@ugent.be<br />

Abstract: Starting a new business is often d<strong>on</strong>e in a realm of improvisati<strong>on</strong> if resources are scarce and the<br />

business horiz<strong>on</strong> is far from clear. Strategic improvisati<strong>on</strong> occurs when the design of novel activities unite. We<br />

c<strong>on</strong>ducted an investigati<strong>on</strong> of so called ‘emergent collectives’ in the c<strong>on</strong>text of a small and medium-sized<br />

enterprise (SME). Emergent collectives are networks of informati<strong>on</strong> nodes with minimal central c<strong>on</strong>trol and<br />

largely c<strong>on</strong>trolled by a protocol specificati<strong>on</strong> where people can add nodes to the network and have a social<br />

incentive to do so. We c<strong>on</strong>sidered here emergent collectives around an enterprise resources planning (ERP)<br />

software and a customer relati<strong>on</strong> management (CRM) software in two open source software (OSS) communities.<br />

We investigated how the use of bricolage in the c<strong>on</strong>text of a start-up microenterprise can facilitate the adopti<strong>on</strong> of<br />

an informati<strong>on</strong> system (IS) based <strong>on</strong> emergent collectives. Bricolage is an improvisati<strong>on</strong>al approach that allows<br />

learning form c<strong>on</strong>crete experience. In our case study we followed the incepti<strong>on</strong> of a new business initiative up to<br />

the implementati<strong>on</strong> of an IS, during a period of two years. The case study covers both the usefulness of bricolage<br />

for strategic improvisati<strong>on</strong> and for entrepreneurial activity in a knowledge-intensive new business. We adopted an<br />

interpretative research strategy and used participatory acti<strong>on</strong> research to c<strong>on</strong>duct our inquiry. Our findings lead to<br />

the suggesti<strong>on</strong> that emergent collectives can be moulded into a usable set of IS resources applicable in a<br />

microenterprise. However the success depends heavily <strong>on</strong> the ICT managerial and technological capabilities of<br />

the CEO and his individual commitment to the process of bricolage. Our findings also show that open ERP and<br />

CRM software are not passing delusi<strong>on</strong>s. These emergent collectives will not take over proprietary ERP and<br />

CRM software all of a sudden, but clearly the rules of the game are slowly changing due to the introducti<strong>on</strong> of<br />

new business models. The study c<strong>on</strong>tributes to the research of OSS as emergent collectives, bricolage and IS<br />

adopti<strong>on</strong> in SMEs.<br />

Keywords: SMEs, bricolage, emergent collectives, open software, ERP, CRM, IS adopti<strong>on</strong><br />

1. Introducti<strong>on</strong><br />

Starting a new business is often d<strong>on</strong>e in a realm of improvisati<strong>on</strong> if resources are scarce and the<br />

business horiz<strong>on</strong> is far from clear. Start-ups and small- and medium-sized enterprises (SMEs) often<br />

adopt informati<strong>on</strong> technology (IT) and informati<strong>on</strong> systems (IS) in order to facilitate a start-up.<br />

However, adopting IT/IS into an embry<strong>on</strong>ic organizati<strong>on</strong>al structure with a lack of rigid business<br />

processes is a complex and risky task. Many investments in IT/IS, such as Enterprise Resource<br />

Planning (ERP) or Customer Relati<strong>on</strong> <strong>Management</strong> (CRM), outsourced as well as in-sourced, never<br />

fully reach the intended objectives and are therefore c<strong>on</strong>sidered as not being successful. Despite our<br />

knowledge of IT/IS implementati<strong>on</strong>, a lot of IT projects still fail (Avis<strong>on</strong> et al. 2006, Devos et al. 2008,<br />

Bharadwaj et al. 2009, Group 2004). Past and recent research has also revealed that SMEs tend to<br />

lean str<strong>on</strong>gly <strong>on</strong> external expertise for IT adopti<strong>on</strong> (Th<strong>on</strong>g et al. 1996, Dibbern and Heinzl 2009). IT<br />

outsourcing greatly increases the complexity of governing these endeavours and brings in new risks<br />

and burdens for IS success (Aubert et al. 2005). Although SMEs have specific characteristics like<br />

organizati<strong>on</strong>al flexibility, limited span of c<strong>on</strong>trol and fast decisi<strong>on</strong>-making, the development of internal<br />

resources and capabilities for IS adopti<strong>on</strong> is still a critical problem because SMEs are resource<br />

c<strong>on</strong>straint (Raym<strong>on</strong>d 1985).<br />

In this work we intend to build <strong>on</strong> and move bey<strong>on</strong>d existing work to provide c<strong>on</strong>ceptual<br />

underpinnings for the study of bricolage applied for adopting IT in a start-up enterprise. Bricolage is<br />

an improvisati<strong>on</strong>al approach that allows learning form c<strong>on</strong>crete experience. We highlight the tensi<strong>on</strong><br />

between the dominant view of classical governance models rooted in c<strong>on</strong>trol theory and the<br />

alternative approach of bricolage. We discuss both the c<strong>on</strong>cept of bricolage and how a bricolagebased<br />

arrangement might be used into the organizati<strong>on</strong>al c<strong>on</strong>text of an SME. We adopted an<br />

interpretative research strategy and used participatory acti<strong>on</strong> research (PAR) to carry out our inquiry.<br />

82


Jan Devos, Hendrik Van Landeghem and Dirk Deschoolmeester<br />

We c<strong>on</strong>ducted an investigati<strong>on</strong> of emergent collectives in the c<strong>on</strong>text of a start-up. Emergent<br />

collectives are networks of informati<strong>on</strong> nodes with minimal central c<strong>on</strong>trol and largely c<strong>on</strong>trolled by a<br />

protocol specificati<strong>on</strong> where people can add nodes to the network. Petrie (2011) refers to an<br />

emergent collective as an ant col<strong>on</strong>y in which its behaviour, and intelligence is the result of the rather<br />

mindless interacti<strong>on</strong>s of individual ants following simple protocols of interacti<strong>on</strong> that result in<br />

qualitatively different global behaviour (Petrie 2011). The motivati<strong>on</strong> for collectively acting lays within<br />

the capacity of the networks to scale and to increase value for the user. We c<strong>on</strong>sidered here the<br />

emergent collectives around an ERP software and a CRM software in two open source software<br />

(OSS) communities. We formulated our research questi<strong>on</strong> as: how can the use of bricolage facilitate<br />

the adopti<strong>on</strong> of emergent collectives in an entrepreneurial setting? In a real life case we followed the<br />

incepti<strong>on</strong> of a new business initiative up to the implementati<strong>on</strong> of an IS during a period of two years.<br />

The case study covers both the usefulness of bricolage for strategic improvisati<strong>on</strong> and for the<br />

entrepreneurial activity in a knowledge-intensive new business.<br />

This paper is structured in five main secti<strong>on</strong>s, starting with this introducti<strong>on</strong>. In the following secti<strong>on</strong><br />

we review the recent literature <strong>on</strong> bricolage and IT. In the third secti<strong>on</strong> we elaborate <strong>on</strong> our research<br />

methodology based <strong>on</strong> acti<strong>on</strong> research and we introduce the case study. In fourth secti<strong>on</strong> we bring<br />

the findings of our inquiries. Secti<strong>on</strong> five discusses the c<strong>on</strong>clusi<strong>on</strong>s and implicati<strong>on</strong>s of our work for<br />

academics and practiti<strong>on</strong>ers.<br />

2. Bricolage<br />

The c<strong>on</strong>cept of bricolage was introduced by the French anthropologist Lévi-Strauss in his book, La<br />

pensée sauvage published in 1962 and translated in English to The Savage Mind (Lévi-Strauss<br />

1968). Bricolage is Lévi-Strauss’s term to describe the mythical thinking of primitive people, who used<br />

a fixed set of ideas that they combined and recombined in different ways (Pohn 2003-2007). The word<br />

bricolage is French and does not have a precise equivalent in English. It can be translated as<br />

tinkering or playing/messing around. Lévy-Strauss uses bricolage as an analogy to spell out the<br />

processes underlying mythical thoughts (Duymedjian and Ruling 2010). The bricoleur is the<br />

handyman, tinkerer or do-it-yourselfer. It can be noticed that the words bricolage and bricoleur applies<br />

to playing and refers to devious acti<strong>on</strong>s. Lévi-Strauss (1968) compares bricolage as ‘the science of<br />

the c<strong>on</strong>crete’ as opposed to logical thinking grounded in (positivistic) science and characterized<br />

bricolage as a particular way of acting as ‘doing things with whatever is at hand’. The science of the<br />

c<strong>on</strong>crete is characterized by a c<strong>on</strong>cern for exhaustive observati<strong>on</strong>, systematic inventorying of all<br />

elements and relies <strong>on</strong> a highly developed mode of understanding based <strong>on</strong> the intimacy with the<br />

c<strong>on</strong>crete (Duymedjian and Ruling 2010). The bricoleur is not a craftsman and bricolage does not<br />

proceed in a straightforward, linear and rati<strong>on</strong>al way. Instead bricolage wander from <strong>on</strong>e thing to<br />

another and has a fragmented nature reflecting its affinity with play (Pohn 2003-2007). Being a<br />

bricoleur also means being a thinker tinkerer with focus <strong>on</strong> instant objects and materials at hand to<br />

approach soluti<strong>on</strong>s for problems faced (Coleman 2006). Bricolage is not very well articulated as a<br />

theory. Lévi-Strauss describes the process of bricolage through the role descripti<strong>on</strong> of the bricoleur. In<br />

a dichotomous category the bricoleur is the opposed ideal-type of the engineer. From the seminal<br />

work of Lévi-Strauss, three c<strong>on</strong>structs can be inferred to characterize bricolage: 1) repertoire or the<br />

material and immaterial resources that are collected independently of any particular project or<br />

utilizati<strong>on</strong>, 2) dialogue or the activity of assembling objects and 3) outcome, which’s refers both to the<br />

process and its results (Duymedjian and Ruling 2010). Bricolage is related to improvisati<strong>on</strong>,<br />

sensemaking, entrepreneurship and the work of technical systems (Duymedjian and Ruling 2010).<br />

Bricolage was introduced in anthropology and found its way into cognitive sciences, Informati<strong>on</strong><br />

Technology (Ferneley and Bell 2006, Johri 2011, DesAutels 2011, Ciborra 2002), Entrepreneurship<br />

(Phillips and Tracey 2007, Baker et al. 2003), Innovati<strong>on</strong> Research (Fuglsang and Sorensen 2011,<br />

Banerjee and Campbell 2009), Informati<strong>on</strong> Sciences (Coleman 2006) and Organizati<strong>on</strong> Theory<br />

(Duymedjian and Ruling 2010, Weick 1998). In this work we elaborate <strong>on</strong> bricolage in IT. Pi<strong>on</strong>eer of<br />

the research <strong>on</strong> bricolage and IT is Claudio Ciborra (Ciborra 2002). Ciborra (2002) criticized the way<br />

strategic thinking about IT in organizati<strong>on</strong>s is often presented as a linear, top-down, rati<strong>on</strong>al and<br />

cognitive process. When put into use by practiti<strong>on</strong>ers strategic planning is a process of disassociati<strong>on</strong><br />

from the theoretical foundati<strong>on</strong>s. The trajectory from IT strategy formulati<strong>on</strong> down to implementati<strong>on</strong> is<br />

not an intenti<strong>on</strong>al process of design, but a chain of evoluti<strong>on</strong>ary processes that involve serendipity<br />

and muddling through elements of surprise. The analysis of Ciborra (2002) is compliant with the<br />

phenomen<strong>on</strong> of emergent collectives (Petrie 2011). His example of the early launch of the Internet is<br />

most compelling: ..[..] ARPANET did not take off as expected and it was far from being an undisputed<br />

83


Jan Devos, Hendrik Van Landeghem and Dirk Deschoolmeester<br />

success. What helped to transform a research network into the full-blown Internet was a myriad of<br />

hacks, surprises, and improvisati<strong>on</strong>s, mostly stemming from the users’ envir<strong>on</strong>ment, and the<br />

benevolent and tolerant ARPA project management practices (Ciborra 2002). Ciborro (2002)<br />

introduces the c<strong>on</strong>cept of bricolage as an alternative for the systematic and procedural way of<br />

organizing and executing work. Bricolage, as opposed to the pre-planned way of operating can be<br />

highly effective since it can fit the c<strong>on</strong>tingencies of the moment. Ciborra (2002) poses that informati<strong>on</strong><br />

systems have a high degree of flexibility in their use making them ideal for bricolage.<br />

The resources at hand for IT bricolage are hardware and software artefacts. The IT bricoleur interacts<br />

with existing software, by redesigning, modifying and adding new functi<strong>on</strong>ality and by doing so, new<br />

ways of using the software are explored. Although it is comm<strong>on</strong> that bricolage is executed <strong>on</strong> a<br />

operati<strong>on</strong>al level, bricolage is also experienced in strategic acti<strong>on</strong>. The IT bricolage approach is very<br />

similar to the activities that can be observed in the emergent collectives of open source software<br />

(OSS) communities (Ferneley and Bell 2006). OSS users as well as developers work intimately<br />

together <strong>on</strong> requirements, try them out with tinkering to the code and in so doing a useful software<br />

can emerge. Examples of such OSSs are the communities of OpenERP (www.openerp.com) and<br />

MAGENTO (www.magento.org) which were used in the case study.<br />

The instantiati<strong>on</strong> of the c<strong>on</strong>cept of bricolage is suggested by seven oxymor<strong>on</strong>’s (Ciborra 2002). These<br />

oxymor<strong>on</strong>’s represent a systematic approach for the establishment of a new organizati<strong>on</strong>al setting<br />

where new systems can be adopted. The paradoxical reflecti<strong>on</strong>s can provoke new ways of thinking<br />

and c<strong>on</strong>siderati<strong>on</strong>. Each of the oxymor<strong>on</strong>’s c<strong>on</strong>stitutes a thinking frame that excludes forms of<br />

established organizati<strong>on</strong>al routines and existing c<strong>on</strong>trol systems. We developed here the oxymor<strong>on</strong>’s<br />

as propositi<strong>on</strong>s of the theory of bricolage.<br />

First oxymor<strong>on</strong> is value bricolage strategically (VBS). The status of bricolage in an organizati<strong>on</strong> can<br />

balance between a highly competent behaviour and incompetence. The bricoleur operates in a fuzzy<br />

work z<strong>on</strong>e that offers liberty and experimenting with the choices of which resources at hand will be<br />

used. The soluti<strong>on</strong>s that come out of the process of bricolage need to be embedded in an everyday<br />

experience and local knowledge as well as having a strategic impact. The sec<strong>on</strong>d oxymor<strong>on</strong> is design<br />

tinkering (DT). Prototyping and experimentati<strong>on</strong> must be facilitated through arrangement of activities,<br />

settings and systems. Knowledge is generated through design and by creating acti<strong>on</strong>s, and acti<strong>on</strong>s<br />

are evaluated to build knowledge. Third oxymor<strong>on</strong> is establish systematic serendipity (ESS). A climate<br />

for unexpected soluti<strong>on</strong>s must be provided through the c<strong>on</strong>currency of c<strong>on</strong>cepti<strong>on</strong>, implementati<strong>on</strong>,<br />

and executi<strong>on</strong> that intermingle c<strong>on</strong>stantly. Fourth oxymor<strong>on</strong> is thrive <strong>on</strong> gradual breakthroughs (TGB).<br />

The emerging ideas and soluti<strong>on</strong>s must lead to managerial routines that helps to bring to new institute<br />

to the level of a simple organisati<strong>on</strong>al structure (Mintzberg 1993). Fifth oxymor<strong>on</strong> is practise unskilled<br />

learning (PUL). Unlearning the old ways of thinking and challenging incremental learning while<br />

incorporating the risks of behaving incompetent. Sixth oxymor<strong>on</strong> is strive for failure (SFF). Formative<br />

evaluati<strong>on</strong> of failures can generate new ideas and designs. Striving for excellence is the summative<br />

evaluati<strong>on</strong> of successes and does not lead to innovati<strong>on</strong> or change. Finally the seventh oxymor<strong>on</strong> is<br />

achieve collaborative inimitability (ACI). The activities of bricolage are highly idiosyncratic, often latent<br />

and are not easy to imitate. This can be seen as a vital source for competitive advantage for SMEs to<br />

remain agile and resp<strong>on</strong>sive to the business envir<strong>on</strong>ment the inimitability should be the key for<br />

creating a competitive advantage but collaborati<strong>on</strong>, even with competitors, in developing strategic<br />

applicati<strong>on</strong>s should not be avoided.<br />

3. Research methodology<br />

This research projects aims at two goals: first we have to give an answer to a research questi<strong>on</strong> and<br />

sec<strong>on</strong>d it has to fulfil a business need. Although the last goal is strictly not necessary to acquire<br />

scientific knowledge, it is part of our specific setup and research method. Therefore we adopted for<br />

our investigati<strong>on</strong>s PAR since we were dealing here with a complex social system that cannot be<br />

reduced for a meaning study (Baskerville 1999). Acti<strong>on</strong> research aims to solve current practical<br />

problems while expanding scientific knowledge (Baskerville and Myers 2004). We have worked with<br />

practiti<strong>on</strong>ers in a well chosen case study to solve an important practical problem: the adopti<strong>on</strong> of an<br />

informati<strong>on</strong> system based <strong>on</strong> emergent collectives in an entrepreneurial start-up.<br />

Since PAR was chosen as our research method, this involves that we are taking an interpretive<br />

stance of the research enquiry and that we are not aiming to a broad generalizati<strong>on</strong> of the results.<br />

According to Baskerville (1999) acti<strong>on</strong> research implies the adopti<strong>on</strong> of an idiographic viewpoint. Also<br />

84


Jan Devos, Hendrik Van Landeghem and Dirk Deschoolmeester<br />

the interpretative perspective of the research process, is aiming at making sense of the<br />

phenomen<strong>on</strong>’s under investigati<strong>on</strong>. In the quest for an answer to our research questi<strong>on</strong> we brought<br />

the theory of bricolage to a deeper stage of development and understanding. The use of theory in our<br />

research is threefold: 1) as a guide to design, 2) as an iterative process of data collecti<strong>on</strong> and<br />

analysis, and 3) as a final product (Walsham 2006). The setting up and carrying out of fieldwork is the<br />

fundamental basis of any interpretative study (Walsham 2006, Klein and Myers 1999). All acti<strong>on</strong>s of<br />

the researchers and the CEO were documented into logbooks. The findings of our acti<strong>on</strong>s were<br />

coded out of our descripti<strong>on</strong>s in the logbooks. We used axial coding to relate the c<strong>on</strong>cepts in the<br />

descripti<strong>on</strong>s with the theoretical propositi<strong>on</strong> of the bricolage theory (Corbin and Strauss 2008).<br />

For our research plan we drew <strong>on</strong> the acti<strong>on</strong> research process proposed by Baskerville (1999) and <strong>on</strong><br />

the PAR used in the work of Street and Meister (2004). However we also differentiated our research<br />

plan slightly according to the specific situati<strong>on</strong> we dealt with. Acti<strong>on</strong> research c<strong>on</strong>sist of a cyclic form<br />

of five phases. Figure 1 illustrates the acti<strong>on</strong> research cycle. The acti<strong>on</strong> research cycle can be<br />

performed as many times as needed for achieving a soluti<strong>on</strong> to the problem. We discussed within the<br />

research team <strong>on</strong> the number of cycles and decided to c<strong>on</strong>duct <strong>on</strong>ly two cycles: a baseline analysis<br />

and an implementati<strong>on</strong> cycle. The Client-System Infrastructure c<strong>on</strong>stitutes the agreement of our<br />

research envir<strong>on</strong>ment. The structural acti<strong>on</strong> research cycle starts with the diagnosing phase which<br />

identifies the primary problems and leads to the theoretical assumpti<strong>on</strong>s of newly organizati<strong>on</strong>. In this<br />

phase the researchers interviewed the CEO during several sessi<strong>on</strong> to understand the past, present,<br />

and future use of IT and how the CEO thinks IT could be beneficial for the organizati<strong>on</strong>.<br />

Figure 1: The acti<strong>on</strong> research cycle<br />

The acti<strong>on</strong> planning phase c<strong>on</strong>tains the organizati<strong>on</strong>al acti<strong>on</strong>s that deal with the problems defined in<br />

the previous phase. The next stage is acti<strong>on</strong> taking and implements the planned acti<strong>on</strong>s. The<br />

interventi<strong>on</strong> of the researchers is n<strong>on</strong>-directive. The change is sought indirectly in process of cut-andtry.<br />

The test of the theoretical assumpti<strong>on</strong>s was d<strong>on</strong>e in the evaluating phase. The last phase is the<br />

specific learning phase in which less<strong>on</strong>s learned are derived.<br />

Case Study Endoxa<br />

Endoxa is a Greek word and was used by Aristotle to acknowledge the tested beliefs of a community.<br />

The CEO of Endoxa discovered a business opportunity which is similar to its existing operati<strong>on</strong>s. It is<br />

obvious that new e-business initiatives, but also existing <strong>on</strong>es, suffer largely from a shortage of<br />

logistic capabilities. In this new venture, Endoxa is aiming at becoming more than a drop shipping<br />

agent, but rather as an network orchestrator of the complete cross chain supply of delivering products<br />

to customers. To build the necessary supporting and enabling business processes for this new<br />

attempt, Endoxa adopts the visi<strong>on</strong> of emergent collectives of OSS and will combine existing<br />

technologies into a new strategic informati<strong>on</strong> system. Although Endoxa is a very small<br />

entrepreneurial enterprise it is compliant with the five criteria defined by Mintzberg, to c<strong>on</strong>stitute a<br />

85


Jan Devos, Hendrik Van Landeghem and Dirk Deschoolmeester<br />

minimal organizati<strong>on</strong> (Mintzberg 1983). There is direct supervisi<strong>on</strong>, little formalized behaviour, an<br />

organic structure, a strategy planning and the CEO formulate plans intuitively as an extensi<strong>on</strong> of his<br />

own pers<strong>on</strong>ality.<br />

Before the research project took off, the CEO of Endoxa was already involved in another research<br />

project <strong>on</strong> the nature of emergent collectives. As a serial entrepreneur the CEO was in search for<br />

assistance to see how OSS could be of use to start a new company. As researchers we had the<br />

chance to observe the take off of a new enterprise and this offered an excellent opportunity to enlarge<br />

our understanding of how IT/IS can be of critical importance in organisati<strong>on</strong>s. An agreement for a<br />

research partnership was formalized that stipulated the rights and obligati<strong>on</strong>s of the researchers and<br />

the CEO and his collaborators. The acti<strong>on</strong>s of the researchers were performed in an open way and<br />

were aiming at a beneficial impact <strong>on</strong> the organizati<strong>on</strong>. All acti<strong>on</strong>s were d<strong>on</strong>e in close harm<strong>on</strong>y with<br />

the CEO. The CEO provides the necessary knowledge to the researchers for being harvested in an<br />

academic inquiry. However both parties had their objectives. It was clear from the beginning that the<br />

CEO was aiming at a fruitful start-up for his enterprise. The objectives of the researchers were spelled<br />

out in the research questi<strong>on</strong>. The design of the artefact is for the researchers is <strong>on</strong>ly a means to an<br />

end. The theory or c<strong>on</strong>cept of bricolage was explained to the CEO, however the c<strong>on</strong>cept of emergent<br />

collectives was very well known to the CEO.<br />

4. Findings<br />

First it was noticed that the stages of the acti<strong>on</strong> cycles do not always proceed in linear and straight<br />

forward way. The appearance of the organisati<strong>on</strong>al acti<strong>on</strong>s is c<strong>on</strong>current and not always<br />

synchr<strong>on</strong>ised. It was the task of the researchers to shed light <strong>on</strong> the different acti<strong>on</strong>s and make an<br />

appropriate analysis of the findings. We summarized our findings in table 1, showing the findings<br />

during the baseline analysis cycle and in table 2, showing the findings during the Implementati<strong>on</strong><br />

cycle.<br />

During the first team meetings with the CEO ten acti<strong>on</strong>s were identified, starting from an overall<br />

company missi<strong>on</strong> that was spelled out in a proposal (BRIDEE) and submitted to a business school.<br />

The overall company missi<strong>on</strong> was detailed in a competitor’s analysis, a market research plan, a<br />

business and a financial model. Two specific acti<strong>on</strong>s, e-AirwayBill and polling quotes Fedex were<br />

defined as nice-to-haves but it was in no way sure that the implementati<strong>on</strong> was feasible within<br />

Endoxa. The choice for OpenERP and Magento was fixed and the organisati<strong>on</strong>al modelling was d<strong>on</strong>e<br />

with these two products as missi<strong>on</strong> critical systems. OpenERP was suggested as the back-office<br />

soluti<strong>on</strong> and Magento as the fr<strong>on</strong>t-office soluti<strong>on</strong>. Finally a project was set up for the acquirement of<br />

government subsidies for a new start-up initiative.<br />

For the acti<strong>on</strong> planning an agenda was set up to work <strong>on</strong> each of the acti<strong>on</strong>s during two days a week.<br />

The researchers got mingled with the CEO and the collaborators of Endoxa and for each acti<strong>on</strong> a<br />

planning was made. The acti<strong>on</strong> planning and acti<strong>on</strong> taking are also shown in table 1. The theory<br />

building is mainly inferred from the evaluating and learning phases. For the bottom line we found five<br />

acti<strong>on</strong>s that were classified as Value Bricolage Strategically (VBS). These fives acti<strong>on</strong>s, the Missi<strong>on</strong><br />

Statement, the Competitors Analysis, the Market Research Plan, the Business and the Financial<br />

Model can be seen as the transformed stakeholders needs into an enterprise’s acti<strong>on</strong>able strategy.<br />

However the down-top translati<strong>on</strong> is not feasible into specific goals at every area of the enterprise.<br />

The acti<strong>on</strong>s were c<strong>on</strong>sidered as strategic but still subject to modificati<strong>on</strong>s and adaptati<strong>on</strong>s.<br />

Two specific acti<strong>on</strong>s, e-AirwayBill and Polling Quotes Fedex were classified as Design Tinkering (DT)<br />

and Practise Unskillful Learning (PUL). It was designing something that was already in use. The<br />

less<strong>on</strong> learned was that it is better to use what is already build and proven than to make something<br />

new. This is actually compliant with the resource c<strong>on</strong>strains in SMEs.<br />

The acti<strong>on</strong>s OpenERP and Magento where of major importance for Endoxa and there was a tense<br />

force coming from the CEO to strive for a breakthrough. This illustrates that an ERP system is missi<strong>on</strong><br />

critical in SMEs and perceived as such by the CEO. The basic business processes like invoicing,<br />

general ledger, accounts receivable and payable as well as the more strategic processes like<br />

tendering, sales, order entry and bidding need to come together in <strong>on</strong>e integrated system. Proprietary<br />

ERP software was not an opti<strong>on</strong> for Endoxa, because of the costs. The acti<strong>on</strong>s were c<strong>on</strong>sidered to be<br />

a match with the oxymor<strong>on</strong> of Thrive for Gradual Breakthroughs (TGB). The strive for subsidies was<br />

already c<strong>on</strong>sidered by the CEO as not feasible due to a shortage of manpower and administrative<br />

86


Jan Devos, Hendrik Van Landeghem and Dirk Deschoolmeester<br />

agility. Still the acti<strong>on</strong> was kept open in the hope that a file could be submitted for a positive<br />

evaluati<strong>on</strong>. This acti<strong>on</strong> was classified as a Thrive for Failure (TFF). The end of the first cycle and the<br />

start of the sec<strong>on</strong>d cycle did not follow a linear trajectory. During the baseline cycle, already acti<strong>on</strong>s<br />

were defined for the implementati<strong>on</strong> cycle. The implementati<strong>on</strong> cycle was characterised by much<br />

more diagnosed acti<strong>on</strong>s as can seen in table 2. The acti<strong>on</strong>s OpenERP and Magento were c<strong>on</strong>sidered<br />

as the most imported acti<strong>on</strong>s of the cycle and were matched with five oxymor<strong>on</strong>’s of bricolage: VBS,<br />

DT, ACI, SFF and PUL.<br />

5. C<strong>on</strong>clusi<strong>on</strong>s<br />

In this paper we present the findings from a participatory acti<strong>on</strong> research describing how bricolage<br />

can facilitate the adopti<strong>on</strong> of emergent collectives in the form of OSS in a microenterprise. The use of<br />

the PAR helped us to make our research more relevant to practice. We argue that our work differs<br />

with that of c<strong>on</strong>sultants. Our theoretical perspective of bricolage was made clear in advance and<br />

before any acti<strong>on</strong> was taken in the organizati<strong>on</strong>. We mapped the practical acti<strong>on</strong>s with the<br />

propositi<strong>on</strong>s of the theory of bricolage, operati<strong>on</strong>alized by the oxymor<strong>on</strong>’s. The relati<strong>on</strong>ships between<br />

the elements of created artefact are made more visible then previously during the acti<strong>on</strong>s. Our<br />

understanding of the c<strong>on</strong>structs of bricolage has been increased.<br />

Our research has revealed the pivotal roles of the CEO in which IT/IS is adopted and implemented. A<br />

positive attitude of the CEO towards IT/IS was noticeable during the entire investigati<strong>on</strong> period. This is<br />

compliant with previous research <strong>on</strong> the role of the CEO and the adopti<strong>on</strong> of IT/IS (Cragg and King<br />

1993, Th<strong>on</strong>g et al. 1996). However different roles of the CEO could be observed: first of all the role of<br />

the owner-manager which always kept a sharp look <strong>on</strong> the profitability of the endeavour and the<br />

strategic focus. Sec<strong>on</strong>dly the role of an individual high-end user who was intimately involved into the<br />

daily use of the software in all the implemented business processes. Thirdly the role of CIO and IT<br />

manager who steered the project of bricolage and utilized the mechanism of IT project management<br />

like organizing steering committees meetings, communicati<strong>on</strong>s sessi<strong>on</strong> for the users, and<br />

documenting the acti<strong>on</strong>s and realizati<strong>on</strong>s of the project members.<br />

From our findings it could be noticed that the process of bricolage was sometimes getting in the way<br />

of the daily business operati<strong>on</strong>s. Since Endoxa is a start-up this was not so important but this<br />

indicates that the process of bricolage need to come to a moderate intensity to reduce the<br />

organisati<strong>on</strong>al turbulence and to refocus <strong>on</strong> organisati<strong>on</strong>al efficiency. It was already noticed by<br />

Ferneley et al. (2006) that IS bricolage is not without its dangers: “the entropy of the IS can increase<br />

as changes are made, rendering the IS architecture unmanageable and inefficient”. Bricoleurs and<br />

certainly entrepreneurial bricoleurs have to keep in mind that the process of bricolage should take<br />

place within the boundaries of a minimal organisati<strong>on</strong>al structure (Weick 1993). Also at a certain point<br />

after the change process is established, a phase of entropy-reducing is needed to allow the new<br />

systems to take off and to fade away the organisati<strong>on</strong>al turbulence.<br />

It has been shown that OSS in ERP and CRM type applicati<strong>on</strong> domains, where c<strong>on</strong>venti<strong>on</strong>al wisdom<br />

says it is impossible to design from an open software perspective, holds a valuable promise. Many<br />

software project leaders would not dare to choose for OSS in an entrepreneurial setting and would<br />

prefer propriety software stating that the quality of the latter <strong>on</strong>e is far more superior than open<br />

source. Although we did not investigate that statement, in our empirical findings we found evidence<br />

that the development of an informati<strong>on</strong> system with OSS is certainly not a straightforward process,<br />

nor that the development process if free from errors and flaws, but this is no way other than for<br />

propriety software. By choosing for OSS the SMEs has avoided the vendor lock inn that comes all too<br />

often with the adopti<strong>on</strong> of propriety software and has reduced the total cost of ownership of the<br />

informati<strong>on</strong> system. A rough estimate has revealed that the cost of implementati<strong>on</strong> of OpenERP<br />

comes to the same high as a feasibility study for a mainstream propriety ERP vendor.<br />

Table 1: The baseline analysis cycle<br />

Diagnosing Acti<strong>on</strong> Planning Acti<strong>on</strong> Taking Evaluating Specifying Learning<br />

1 BRIDEE Spelling out the<br />

missi<strong>on</strong><br />

statement?<br />

Submitting for<br />

a business<br />

school<br />

competiti<strong>on</strong>.<br />

87<br />

VBS Missi<strong>on</strong> statement needs<br />

refinement but is not mandatory<br />

for a bottom up ‘bricolage’<br />

approach.


2 Competitors<br />

Analysis<br />

Jan Devos, Hendrik Van Landeghem and Dirk Deschoolmeester<br />

List of three direct<br />

competitors was<br />

edited (big 3):<br />

Shipwire,<br />

Shipworks and<br />

Easyshipping.<br />

Investigati<strong>on</strong><br />

of the support<br />

of the web<br />

shop<br />

platforms of<br />

each the ‘big<br />

3’.<br />

VBS The obtained informati<strong>on</strong> was<br />

used as a benchmark for the<br />

own realizati<strong>on</strong>s.<br />

Diagnosing Acti<strong>on</strong> Planning Acti<strong>on</strong> Taking Evaluating Specifying Learning<br />

3 Market What are the Offering a VBS To fuzzy to be of real value<br />

Research questi<strong>on</strong>s that platform for<br />

Plan web shops have the support of<br />

c<strong>on</strong>sidering their the logistic<br />

logistic processes of<br />

processes. web shops<br />

4 Business A sound business Refining and VBS To fuzzy to be of real value<br />

Model<br />

model adapting the<br />

business<br />

model to<br />

current<br />

insights and<br />

developments<br />

5 Financial Calculating Comparing VBS To fuzzy to be of real value<br />

Model financial flows, the figures<br />

cash flows, OPEX with partners<br />

and CAPEX. and<br />

competitors<br />

6 e-AirwayBill Visualize the Testing DT, PUL Never build what is already build<br />

XML-structure of<br />

the transport<br />

documents.<br />

by others.<br />

7 polling quotes How to use Web Test account DT, PUL Never build what is already build<br />

Fedex Services in<br />

logistic processes<br />

with Fedex<br />

by others.<br />

8 OpenERP All business Adopting the TGB The CEO had a str<strong>on</strong>g belief in<br />

processes should full set of<br />

open software products and the<br />

be implemented functi<strong>on</strong>s of<br />

use of OpenErp was mandatory.<br />

in OpenERP OpenERP<br />

9 Magento All logistics Adopting the TGB The CEO had a str<strong>on</strong>g belief in<br />

processes of the full set of<br />

open software products and the<br />

web shops should functi<strong>on</strong>s of<br />

use of Magento was mandatory.<br />

be implemented<br />

in Magento.<br />

Magento<br />

10 Strive for Research <strong>on</strong> Try to work SFF Subsidies programs are not<br />

Subsidies three levels: through the<br />

easy accessible for SMEs. The<br />

Regi<strong>on</strong>al,<br />

rigor<br />

bureaucratic burden is to heavy.<br />

Nati<strong>on</strong>al and government<br />

<str<strong>on</strong>g>European</str<strong>on</strong>g>. procedures for<br />

subsidies<br />

Table 2: The implementati<strong>on</strong> cycle<br />

Diagnosing Acti<strong>on</strong> Planning Acti<strong>on</strong> Taking Evaluating Specifying<br />

Learning<br />

1 OpenERP Scheduling of Mailers Feasible soluti<strong>on</strong> found DT OpenERP offers<br />

Versi<strong>on</strong> upgrading<br />

in OpenERP<br />

ACI a soluti<strong>on</strong><br />

Upgrade v.6.0.3 to v.6.1<br />

The upgrade to<br />

was successfully<br />

the latest versi<strong>on</strong><br />

Geotags<br />

implemented<br />

SFF is a pi<strong>on</strong>eering<br />

activity and is not<br />

EBay module, Inventory Module is available, PUL yet followed by<br />

<strong>Management</strong>,<br />

however not stable<br />

most competitors<br />

Warehousing<br />

DT Not all fancy<br />

Extract Transfer Load Module is available, VBS tools are needed<br />

User and access rights deployment is put <strong>on</strong><br />

and useful<br />

The company ‘OpenERP’<br />

hold<br />

Implement<br />

Resources are<br />

c<strong>on</strong>straint<br />

88


Jan Devos, Hendrik Van Landeghem and Dirk Deschoolmeester<br />

Follow up of the<br />

company ‘OpenERP’:<br />

company visits.<br />

Design up to a<br />

workable system<br />

The SME<br />

organisati<strong>on</strong> is<br />

still dependent of<br />

the evoluti<strong>on</strong> of<br />

the opener OSS.<br />

Diagnosing Acti<strong>on</strong> Planning Acti<strong>on</strong> Taking Evaluating Specifying<br />

Learning<br />

2 Magento MagentoERPC<strong>on</strong>nect Installing, c<strong>on</strong>figuring DT Assurance that<br />

(c<strong>on</strong>necti<strong>on</strong> with<br />

and testing<br />

Magento is of<br />

OpenERP)<br />

Dropshipping scenario<br />

use<br />

3 polling Feasibility study Obtain shipping quotes DT, VBS C<strong>on</strong>necti<strong>on</strong> to<br />

quotes<br />

from Fedex<br />

Fedex is of<br />

Fedex<br />

strategic<br />

importance<br />

4 Operati<strong>on</strong>s Daily routines &<br />

Deploy DT Operati<strong>on</strong>s can<br />

procedures for backup and<br />

be implemented<br />

recovery<br />

in a OSS<br />

envir<strong>on</strong>ment<br />

5 IceCAT Feasibility study Installing and testing SFF A lot of offerings<br />

in a OSS are of<br />

no use<br />

6 Bista Alternative for the module Investigate the feasibility DT, VBS Multiple sourcing<br />

Soluti<strong>on</strong>s of drop shipping in<br />

for the<br />

OpenERP<br />

acquisiti<strong>on</strong> of IT<br />

7 Wiki Documentati<strong>on</strong> tool for the Structuring is needed to PUL Documentati<strong>on</strong> is<br />

tools in the repertoire create real value<br />

real problem for<br />

Not for the business<br />

IS<br />

processes:<br />

implementati<strong>on</strong><br />

The documentati<strong>on</strong> for the<br />

business processes should<br />

be into OpenERP<br />

projects.<br />

8 Competiti<strong>on</strong> ShipWire<br />

C<strong>on</strong>stant focus <strong>on</strong> their VBS Keep up with the<br />

ShipEasy<br />

activities<br />

pace of the<br />

ShipWorks<br />

competitors<br />

References<br />

Aubert, B. A., Patry, M. and Rivard, S. (2005) 'A Framework for Informati<strong>on</strong> Technology Outsourcing Risk<br />

<strong>Management</strong>', The DATA BASE for Advances in Informati<strong>on</strong> Systems, 36(4), 9-28.<br />

Avis<strong>on</strong>, D., Gregor, S. and Wils<strong>on</strong>, D. (2006) 'Managerial IT unc<strong>on</strong>sciousness', Communicati<strong>on</strong>s of the Acm,<br />

49(7), 89-93.<br />

Baker, T., Miner, A. S. and Eesley, D. T. (2003) 'Improvising firms: bricolage, account giving and improvisati<strong>on</strong>al<br />

competencies in the founding process', Research Policy, 32(2), 255-276.<br />

Banerjee, P. M. and Campbell, B. A. (2009) 'Inventor bricolage and firm technology research and development',<br />

R & D <strong>Management</strong>, 39(5), 473-487.<br />

Baskerville, R. (1999) 'Investigating Informati<strong>on</strong> Systems with Acti<strong>on</strong> Research', Communicati<strong>on</strong>s of the<br />

Associati<strong>on</strong> for Informati<strong>on</strong> Systems, 2(1), 32.<br />

Baskerville, R. and Myers, M. D. (2004) 'Special Issue <strong>on</strong> Acti<strong>on</strong> Research in Informati<strong>on</strong> Systems: Making is<br />

Research Relevant to Practice - Foreword', Mis Quarterly, 28(3), 329-335.<br />

Bharadwaj, A., Keil, M. and Mahring, M. (2009) 'Effects of informati<strong>on</strong> technology failures <strong>on</strong> the market value of<br />

firms', Journal of Strategic Informati<strong>on</strong> Systems, 18(2), 66-79.<br />

Ciborra, C. (2002) The Labyrinths of Informati<strong>on</strong>: Challenging the Wisdom of Systems, Oxford University Press,<br />

USA.<br />

Coleman, A. S. (2006) 'William Stets<strong>on</strong> Merrill and bricolage for informati<strong>on</strong> studies', Journal of Documentati<strong>on</strong>,<br />

62(4), 462-481.<br />

Corbin, J. and Strauss, A. (2008) Basics of Qualitative Research 3e, Thousand Oaks, California: Sage<br />

Publicati<strong>on</strong>s.<br />

Cragg, P. B. and King, M. (1993) 'SMALL-FIRM COMPUTING - MOTIVATORS AND INHIBITORS', Mis<br />

Quarterly, 17(1), 47-60.<br />

89


Jan Devos, Hendrik Van Landeghem and Dirk Deschoolmeester<br />

DesAutels, P. (2011) 'UGIS: Understanding the nature of user-generated informati<strong>on</strong> systems', Business<br />

Horiz<strong>on</strong>s, 54(3), 185-192.<br />

Devos, J., Van Landeghem, H. and Deschoolmeester, D. (2008) 'Outsourced Informati<strong>on</strong> Systems Failures in<br />

SMEs: a Multiple Case Study', Electr<strong>on</strong>ic Journal of Informati<strong>on</strong> Systems Evaluati<strong>on</strong>, 11(2), 73-84.<br />

Dibbern, J. and Heinzl, A. (2009) 'Outsourcing of Informati<strong>on</strong> Systems Functi<strong>on</strong>s in Small and Medium Sized<br />

Enterprises: A Test of a Multi-Theoretical Model', Business & Informati<strong>on</strong> Systems Engineering, 1(1), 101-<br />

110.<br />

Duymedjian, R. and Ruling, C. C. (2010) 'Towards a Foundati<strong>on</strong> of Bricolage in Organizati<strong>on</strong> and <strong>Management</strong><br />

Theory', Organizati<strong>on</strong> Studies, 31(2), 133-151.<br />

Ferneley, E. and Bell, F. (2006) 'Using bricolage to integrate business and informati<strong>on</strong> technology innovati<strong>on</strong> in<br />

SMEs', Technovati<strong>on</strong>, 26(2), 232-241.<br />

Fuglsang, L. and Sorensen, F. (2011) 'The balance between bricolage and innovati<strong>on</strong>: management dilemmas in<br />

sustainable public innovati<strong>on</strong>', Service Industries Journal, 31(4), 581-595.<br />

Group, S. (2004) Third Quarter Research Report, The Standish Group Internati<strong>on</strong>al.<br />

Johri, A. (2011) 'Sociomaterial bricolage: The creati<strong>on</strong> of locati<strong>on</strong>-spanning work practices by global software<br />

developers', Informati<strong>on</strong> and Software Technology, 53(9), 955-968.<br />

Klein, H. K. and Myers, M. D. (1999) 'A set of principles for c<strong>on</strong>ducting and evaluating interpretive field studies in<br />

informati<strong>on</strong> systems', Mis Quarterly, 23(1), 67-93.<br />

Lévi-Strauss, C. (1968) The Savage Mind, University Of Chicago Press (September 15, 1968).<br />

Mintzberg, H. (1983) Structure in Fives: Designing Effective Organzati<strong>on</strong>s, Englewood Cliffs, NJ: Prentice Hall.<br />

Mintzberg, H. (1993) Structure in Fives: Designing Effective Organizati<strong>on</strong>s, Prentice Hall.<br />

Petrie, C. (2011) 'Emergent Collectives', Ieee Internet Computing, 15(5), 99-102.<br />

Phillips, N. and Tracey, P. (2007) 'Opportunity recogniti<strong>on</strong>, entrepreneurial capabilities and bricolage: c<strong>on</strong>necting<br />

instituti<strong>on</strong>al theory and entrepreneurship in strategic organizati<strong>on</strong>', Strategic Organizati<strong>on</strong>, 5(3), 313-320.<br />

Pohn, K. (2003-2007) 'Cosmicplay.net', [<strong>on</strong>line], available: http://www.cosmicplay.net [accessed<br />

Raym<strong>on</strong>d, L. (1985) 'ORGANIZATIONAL CHARACTERISTICS AND MIS SUCCESS IN THE CONTEXT OF<br />

SMALL BUSINESS', Mis Quarterly, 9(1), 37-52.<br />

Th<strong>on</strong>g, J. Y. L., Yap, C. S. and Raman, K. S. (1996) 'Top management support, external expertise and<br />

informati<strong>on</strong> systems implementati<strong>on</strong> in small businesses', Informati<strong>on</strong> Systems Research, 7(2), 248-267.<br />

Walsham, G. (2006) 'Doing interpretive research', <str<strong>on</strong>g>European</str<strong>on</strong>g> Journal of Informati<strong>on</strong> Systems, 15(3), 320-330.<br />

Weick, K. E. (1993) 'THE COLLAPSE OF SENSEMAKING IN ORGANIZATIONS - THE MANN GULCH<br />

DISASTER', Administrative Science Quarterly, 38(4), 628-652.<br />

Weick, K. E. (1998) 'Improvisati<strong>on</strong> as a mindset for organizati<strong>on</strong>al analysis', Organizati<strong>on</strong> Science, 9(5), 543-555.<br />

90


Determining the Maturity Level of eCommerce in South<br />

African SMEs<br />

David Freeme and Portia Gumede<br />

Rhodes University, Grahamstown, South Africa<br />

d.freeme@ru.ac.za<br />

G08G4290@campus.ru.ac.za<br />

Abstract: According to the United Nati<strong>on</strong>s (2002: 14) report <strong>on</strong> eCommerce adopti<strong>on</strong> and diffusi<strong>on</strong> in South<br />

Africa, South Africa is <strong>on</strong>e of 55 countries at stage 3 of the McKay, Prananto and Marshall’s (2000, 3) model,<br />

namely the interactive stage. This means that it hosts a more sophisticated level of formal interacti<strong>on</strong>s between<br />

users and service providers, via e-mail and post comments. (United Nati<strong>on</strong>s, 2002: 15). Molla and Licker (2004:<br />

91, 92) found that 83% of the SMEs surveyed owned websites. According to the Global Diffusi<strong>on</strong> of the Internet<br />

(GDI) criteria, these figures suggest that SA is at a medium stage of eCommerce maturity, neither immature nor<br />

fully mature (Molla and Licker, 2004: 91, 92). In an attempt to measure the maturity levels of South African SMEs<br />

a checklist, based <strong>on</strong> stages of development models, was developed using six relevant<br />

frameworks/models/classificati<strong>on</strong>s identified using a quantitative research methodology and positivist approach.<br />

The overall finding of the research was that South African SMEs are at stage 2 maturity of the McKay et al,<br />

(2000, 3) model, namely an experimental <strong>on</strong>line presence stage.<br />

Keywords: SME’s, eCommerce, stages of development, web site functi<strong>on</strong>ality, maturity, checklist<br />

1. Introducti<strong>on</strong><br />

The pervasive nature of the internet has changed the way in which countries c<strong>on</strong>duct business<br />

worldwide. Since its emergence in the 1990’s, eCommerce has quickly become the way of c<strong>on</strong>ducting<br />

business <strong>on</strong> a global scale (Cloete, 2002: 2). Of particular interest in the field of eCommerce is its<br />

proliferati<strong>on</strong> in small businesses. In South Africa there are between 1.6 and 3 milli<strong>on</strong> SMEs and they<br />

c<strong>on</strong>tribute 30% to the country’s GDP (Motjolopane and Warden, 2007: 3; Berry et al 2002: 13). There<br />

are as many definiti<strong>on</strong>s for an SME as there are views <strong>on</strong> their characteristics. (Gamage, 2003;<br />

Gilmore, Gallagher and Henry, 2007 and Cloete, 2002b). For the purpose of this paper the definiti<strong>on</strong><br />

of an SME is <strong>on</strong>e that complies with the requirements of the South African Nati<strong>on</strong>al Small Business<br />

Amendment Act, No 26 of 2003. The Act lists a number of requirements that need to be met in order<br />

to be classified as a SME. For the retail and wholesale sector these are: number of employees: 50,<br />

total annual turnover (sector dependent): between R6m to R13m, and total gross asset value (fixed<br />

property exclude): R3m.<br />

The statistical accuracy for any SME study, including the actual number of SMEs in South Africa, is<br />

low as a large number of SMEs, at least three quarters, are in the informal sector of business and so<br />

are essentially legally unrecognized (Berry et al, 2002: 12). Accuracy aside, it is not disputed that<br />

SMEs play a critical role in any ec<strong>on</strong>omy and their ability to c<strong>on</strong>duct eCommerce is of prime<br />

importance to ensure their active inclusi<strong>on</strong> in the new ec<strong>on</strong>omy. The use or level of progressi<strong>on</strong> of<br />

eCommerce in SMEs has typically been studied and adopti<strong>on</strong> encouraged. SMEs generally face<br />

unique and challenging barriers that have inevitably affected the level of adopti<strong>on</strong> or assimilati<strong>on</strong> of<br />

eCommerce into daily operati<strong>on</strong>s. This laggard approach to eCommerce adopti<strong>on</strong> is a characteristic<br />

of developing countries. In South Africa, it was suggested that the available technologies are not<br />

adopted to the extent that is necessary for survival in the current business envir<strong>on</strong>ment (Cloete, 2002:<br />

3; Kruger, 2007: 4).<br />

Research in eCommerce adopti<strong>on</strong> usually c<strong>on</strong>centrates <strong>on</strong> factors that affect adopti<strong>on</strong>, barriers and<br />

stage models (Mohamad and Ismail, 2009: 3, 4). Many maturity models have been proposed for<br />

eCommerce diffusi<strong>on</strong>, with varying approaches and focus points. Mahdi and Steinmueller (2002: 2)<br />

highlighted two classificati<strong>on</strong>s of approaches to determining eCommerce diffusi<strong>on</strong>; the applicati<strong>on</strong><br />

approach and the organizati<strong>on</strong>al approach. The applicati<strong>on</strong> approach delves into the tools of<br />

eCommerce, such as websites and the adopti<strong>on</strong> of informati<strong>on</strong> and communicati<strong>on</strong> technologies<br />

(ICT). The organisati<strong>on</strong>al approach, in c<strong>on</strong>trast, focuses <strong>on</strong> the softer and subjective matters<br />

surrounding the adopti<strong>on</strong> of eCommerce, such as the direct relati<strong>on</strong>ship between the owner’s attitude<br />

towards ICT and the level of adopti<strong>on</strong> and diffusi<strong>on</strong> of eCommerce. The former approach is adopted<br />

for this paper and recognises that as eCommerce activities c<strong>on</strong>tinue to accelerate in the ec<strong>on</strong>omic<br />

envir<strong>on</strong>ment, there is an increasingly important need for scholarly identificati<strong>on</strong> and analysis of the<br />

input factors in the design of a website, to enhance the quality of that website, which is believed to<br />

91


David Freeme and Portia Gumede<br />

positively affect eCommerce diffusi<strong>on</strong> (Chang, Kirk and Litecky, 2001: 125). A quantitative research<br />

methodology was used in this study and a positivist approach employed.<br />

Companies, particularly SMEs, can enter the eCommerce arena at different levels of sophisticati<strong>on</strong>.<br />

Sophisticati<strong>on</strong> refers to the informati<strong>on</strong>, processes, structures and skills adopted by a company for<br />

facilitating transacti<strong>on</strong>s <strong>on</strong>line. Some SMEs enter as ‘brochure-ware’ sites as the first step in creating<br />

a web presence. Others use the internet as a means of c<strong>on</strong>ducting business, taking sales orders<br />

<strong>on</strong>line and processing payments offline. Yet others engage in relatively more complex operati<strong>on</strong>s,<br />

such as offering <strong>on</strong>line catalogues, receiving <strong>on</strong>line orders and handling <strong>on</strong>line payments. Recently<br />

websites have begun to employ newer technologies and features, such as blogs, RSS and alternate<br />

payment processes to enhance the shopping experiences of their customers (Ally, Cater-Steel and<br />

Toleman, 2007: 1009).<br />

The research findings will hopefully propel further research into eCommerce methods enabling policy<br />

makers to recommend soluti<strong>on</strong>s and initiatives to improve South Africa’s current SMEs eCommerce<br />

maturity rating.<br />

2. Research premise<br />

The overall research questi<strong>on</strong> was “What is the eCommerce maturity level of South African SMEs<br />

based <strong>on</strong> their websites informati<strong>on</strong> c<strong>on</strong>tent, functi<strong>on</strong>ality and sophisticati<strong>on</strong>?”<br />

Typically, the initial step is the development of static websites where no prior programming knowledge<br />

is required. The next stage of development provides functi<strong>on</strong>ality which helps in customers’ decisi<strong>on</strong><br />

making (such as order catalogues). As the eCommerce experience increases, databases are<br />

introduced and the website becomes dynamic and increasingly interactive. This is followed by<br />

pers<strong>on</strong>alisati<strong>on</strong>, customizati<strong>on</strong>, search functi<strong>on</strong>s etc. indicating a higher level of sophisticati<strong>on</strong> of a<br />

website (Fisher et al, 2007: 255). Beck, Wigand and K<strong>on</strong>ig (2005: 38) state that websites can be<br />

categorised according to technical measures of what is included in them and how they functi<strong>on</strong>. The<br />

argument being that more sophisticated websites will include applicati<strong>on</strong>s such as email, <strong>on</strong>line<br />

transacti<strong>on</strong> facilities, and customer service or support.<br />

Ally et al, (2007, 1010) proposed that the several “stages of development models” like E-Commerce<br />

Maturity Model (KPMG, 1997), Commitment-Implementati<strong>on</strong> Matrix Model (Stroud, 1998), the<br />

eCommerce Levels (O’C<strong>on</strong>nor and Gavin,1998), Business Lifecycle Model (Berryman, 1999), Intranet<br />

Maturity Model (Dastard and Schemers, 1999), eCommerce Adopti<strong>on</strong> Model (Daniel et.al. 2022) and<br />

the Stages of Growth for business Model (Pregnant et al, 2002), classified a web site by comparing its<br />

functi<strong>on</strong>ality to an eCommerce capability and activity list.<br />

Six relevant frameworks/models/classificati<strong>on</strong>s were identified to determine the most prevalent<br />

functi<strong>on</strong>alities, features and c<strong>on</strong>tent used to evaluate websites of any industry type. These were:<br />

The Centre for Electr<strong>on</strong>ic Commerce (CEC) website evaluati<strong>on</strong> framework.<br />

Model of Internet Commerce Adopti<strong>on</strong> (MICA) (Walcott, 2007),<br />

The extended Model of Internet Commerce Adopti<strong>on</strong> (emic) (Doolin et al, 2002),<br />

Ally et al’s. (2007) 5 stage model,<br />

Doolin et al’s. (2002) 14 levels of functi<strong>on</strong>ality,<br />

Garcia-Boboli et al’s (2005) three web presence classificati<strong>on</strong>s.<br />

Each framework/model/classificati<strong>on</strong> framework/model/checklist was analyzed according to the level<br />

of sophisticati<strong>on</strong>, functi<strong>on</strong>ality, interactivity and complexity of implementati<strong>on</strong> by a c<strong>on</strong>tent element or<br />

c<strong>on</strong>tent feature, shown in Table 1. The premise being that as websites build <strong>on</strong> either complexity or<br />

sophisticati<strong>on</strong>, so the features and functi<strong>on</strong>al comp<strong>on</strong>ents of the site increase (Burgess et al, 2009:<br />

522) indicating the maturity level of eCommerce in SMEs. Ally et al, (2007: 1011) proposed that an<br />

evaluati<strong>on</strong> and assessment of an organisati<strong>on</strong>’s website against a framework will determine the level<br />

at which an organisati<strong>on</strong> currently stands. It indicates the organisati<strong>on</strong>’s maturity at a particular point<br />

in time.<br />

92


David Freeme and Portia Gumede<br />

Table 1: Comparis<strong>on</strong> of functi<strong>on</strong>ality and features and identified Website Checklists<br />

Functi<strong>on</strong>ality/<br />

Feature<br />

CEC MICA eMICA Ally et al Doolin et al<br />

Garcia-Borbolla et<br />

al<br />

Advertising X X X<br />

Order form X X<br />

Online payment X X X X<br />

Offline payment X<br />

Email X X X X X X<br />

Shopping cart X<br />

Minimum security X X X X<br />

Customer<br />

registrati<strong>on</strong> and login<br />

X<br />

Order tracking X X X X<br />

Advanced security X<br />

Customizability X X X<br />

Links X X<br />

Enquiry X X X X<br />

Technical<br />

X<br />

informati<strong>on</strong><br />

Promoti<strong>on</strong> X X X<br />

Catalogue X X X<br />

C<strong>on</strong>tact informati<strong>on</strong> X X<br />

Customer support X X X X<br />

(FAQs, sitemaps)<br />

Chat room,<br />

discussi<strong>on</strong> groups,<br />

blog<br />

X X X<br />

Multimedia X X X X<br />

Feedback, Polls X X<br />

Database search X X X<br />

QuickLinks X X<br />

Pricing informati<strong>on</strong> X X<br />

RSS feed X X<br />

Company X X X<br />

informati<strong>on</strong><br />

Customer policies X<br />

Help functi<strong>on</strong> X<br />

Links to distributors X X<br />

Graphic/Images X X X<br />

Cookies X<br />

Instant Messaging X<br />

History X<br />

3. Website evaluati<strong>on</strong> checklist<br />

A checklist was c<strong>on</strong>structed by comparing and c<strong>on</strong>trasting the differences in the identified quantitative<br />

website frameworks/models/checklists, shown in table 1. The checklist was classified into 3 c<strong>on</strong>tent<br />

areas based <strong>on</strong> our interpretati<strong>on</strong> of how the checklist item relates to the research of Ho (1997) and<br />

Burgess and Cooper (2000):<br />

Static c<strong>on</strong>tent, which is syn<strong>on</strong>ymous to a simple or formal or ornamental presence <strong>on</strong> the web.<br />

Static c<strong>on</strong>tent is informati<strong>on</strong>al in nature and relatively low in sophisticati<strong>on</strong> (largely imitative and<br />

inspired by novelty of innovati<strong>on</strong>) in terms of the programming and technical expertise.<br />

Interactive c<strong>on</strong>tent that groups the features and functi<strong>on</strong>s that support transacti<strong>on</strong>s between the<br />

buyer and the SME, facilitated <strong>on</strong>line.<br />

Transacti<strong>on</strong>al c<strong>on</strong>tent that groups the features and functi<strong>on</strong>s that facilitate communicati<strong>on</strong>, be it<br />

internally (within the SME’s boundaries) or externally.<br />

93


3.1 Static c<strong>on</strong>tent<br />

David Freeme and Portia Gumede<br />

Company Informati<strong>on</strong>: This broad term refers to informati<strong>on</strong> such as the company’s missi<strong>on</strong><br />

statement, financial informati<strong>on</strong>, and history. This is usually found in the secti<strong>on</strong> called ‘about us’ or<br />

the home tab/page of a typical website. Garcia-Borbolla et al, (2005: 175) state that this type of<br />

informati<strong>on</strong> does not have a specific target audience and so serves a promoti<strong>on</strong>al purpose.<br />

C<strong>on</strong>tact Informati<strong>on</strong>: This is c<strong>on</strong>sidered basic informati<strong>on</strong> pertaining to the physical address, email<br />

c<strong>on</strong>tact and other c<strong>on</strong>tact detail, such as teleph<strong>on</strong>e and fax numbers (Doolin et al, 2000). The aim of<br />

this secti<strong>on</strong> is to show the communicative technologies that the SME makes use of.<br />

Pricing Informati<strong>on</strong>: The prices of the products and/or services provided must be present in the<br />

website. This is representative of the SMEs willingness to attract sales from their audience and thus<br />

marks the first step towards <strong>on</strong>line transacti<strong>on</strong>al activities (Garcia-Borbolla et al, 2009: 181; Gwetu,<br />

2005: 63).<br />

Product Catalogue: This may be viewed as a virtual versi<strong>on</strong> of a leaflet/brochure. Promoti<strong>on</strong>al<br />

activities, such as advertising and promoti<strong>on</strong>s (for specials etc.), are encompassed in this feature<br />

(Garcia-Borbolla et al, 2005: 176).<br />

Graphics/Images: This refers to the simple 2 dimensi<strong>on</strong>al graphics and images that support the<br />

aesthetic or design appeal of the website (Fisher et al, 2007: 256).<br />

3.2 Transacti<strong>on</strong>al c<strong>on</strong>tent<br />

Order Form: facilitates <strong>on</strong>line ordering where the user enters all the informati<strong>on</strong> needed to<br />

successfully complete an order.<br />

Online payment opti<strong>on</strong>: a site that has <strong>on</strong>line purchasing capacity to allow payment offline or <strong>on</strong>line.<br />

The latter is more complex and sophisticated to implement and m<strong>on</strong>itor, is now more prevalent with<br />

organisati<strong>on</strong>s especially with the introducti<strong>on</strong> of stringent security measures.<br />

Offline payment opti<strong>on</strong>: This applies when the banking details for payment have been supplied <strong>on</strong> the<br />

site al<strong>on</strong>gside c<strong>on</strong>tact informati<strong>on</strong> to c<strong>on</strong>firm payment. The promoti<strong>on</strong>al activities all take place <strong>on</strong>line<br />

but the final payment step is d<strong>on</strong>e offline.<br />

Shopping cart: This is used to keep the history of the user’s saved purchases and facilitates what is<br />

termed ‘2-click purchasing’ (Elliot et al, 2000: 14).<br />

Security: A policy document or lock <strong>on</strong> transacti<strong>on</strong> data, not <strong>on</strong>ly <strong>on</strong> credit cards are evaluated (Elliot<br />

et al, 2000: 14). There are no distinguishing between advanced and basic security settings in this<br />

study. It is suggested that encrypti<strong>on</strong> and privacy seals should be used to assure security and privacy<br />

of <strong>on</strong>line shopping.<br />

Order tracking: This is an after sale procedure and ensures that the user has order c<strong>on</strong>firmati<strong>on</strong>,<br />

delivery time and trust assurance (Elliot et al, 2000: 14).<br />

3.3 Interactive c<strong>on</strong>tent<br />

Email facilities: the provisi<strong>on</strong> of email facilities as part of the website (for real-time c<strong>on</strong>tact with the<br />

business) is more sophisticated than a static email addresses menti<strong>on</strong>ed in the c<strong>on</strong>tact us page.\<br />

Customer registrati<strong>on</strong> and login: this is a feature found in most sites and it is the threshold for<br />

explorati<strong>on</strong> of many of the other functi<strong>on</strong>al comp<strong>on</strong>ents (pers<strong>on</strong>alizati<strong>on</strong> and access) menti<strong>on</strong>ed in<br />

this proposed framework (Grant and Pears<strong>on</strong>, 2010: 185).<br />

Customisability and pers<strong>on</strong>alizati<strong>on</strong>: This is an enhanced customer service that facilitates<br />

pers<strong>on</strong>alizati<strong>on</strong> of the layout, design, history, features, applicati<strong>on</strong>s etc. (Elliot et al, 2000: 14). The<br />

ability to provide a pers<strong>on</strong>alized, customized interacti<strong>on</strong> for the user allows for website design that<br />

differentiates product and service offerings (Green and Pears<strong>on</strong>, 2010: 186).<br />

94


David Freeme and Portia Gumede<br />

Links to social media sites: This is a new feature introduced in this study. This is in recogniti<strong>on</strong> of the<br />

movement towards <strong>on</strong>line communities and collaborative <strong>on</strong>line tools facilitated by Web 2.0. The<br />

inclusi<strong>on</strong> of links to social media dem<strong>on</strong>strates the recogniti<strong>on</strong> of the marketing opportunities afforded<br />

by social networking. An example would be a link to social networking sites Facebook and LinkedIn<br />

(Gilmore, 2011: 1).<br />

User groups: Chat rooms, blogs, and discussi<strong>on</strong> groups are included in websites in the bid to create<br />

<strong>on</strong>line communities. This can also be used as a form of feedback for the hosts of the websites (Elliot<br />

et al, 2000: 14).<br />

Multimedia: This may be novel or expert use of multimedia items such as audio, video, 3D graphics<br />

and animati<strong>on</strong>s.<br />

Polls: Polls allow users to vote <strong>on</strong> issues and influence the organizati<strong>on</strong>’s decisi<strong>on</strong> making <strong>on</strong> issues<br />

such as policies and service delivery. They are a form of feedback and dem<strong>on</strong>strate a SME’s<br />

willingness to partner with users (Gwetu, 2005: 64).<br />

Search Engine: This enables the user to perform a search for words <strong>on</strong> all site pages while database<br />

search enables queries of informati<strong>on</strong> that exists in the database, an example would be a search for a<br />

specific staff member (Gwetu, 2009: 63).<br />

QuickLinks: These are menu based links that take <strong>on</strong>e to other parts of the site. This is a c<strong>on</strong>venient<br />

navigati<strong>on</strong>al feature in the website (Gwetu, 2009: 63).<br />

RSS Feeds: This can be in the form of news headlines or blog entries. It is a highly interactive feature<br />

(Walcott, 2007: 266).<br />

Help functi<strong>on</strong>: Help serves as a c<strong>on</strong>venience and ease of use tool that improves not <strong>on</strong>ly the<br />

learnability of the website but also user experience (Elliot et al, 2000: 4).<br />

Newsletters: the provisi<strong>on</strong> of electr<strong>on</strong>ic.newsletters.<br />

Links to Distributors: This is c<strong>on</strong>sidered a broader approach to customer services as the idea of<br />

integrati<strong>on</strong> is introduced. The website provides links to upstream or downstream partner websites<br />

(Elliot et al, 2000: 14)<br />

Customer support: Burgess and Cooper (2000) c<strong>on</strong>sider customer support mechanisms, used to<br />

facilitate an improved customer experience, are of medium interactivity in terms maturity levels. FAQs<br />

and sitemaps are examples (Elliot et al, 2000: 14).<br />

4. Sample selecti<strong>on</strong><br />

As this research forms part of an IS H<strong>on</strong>ours dissertati<strong>on</strong>, a sample of 300 randomly selected SMEs’<br />

were identified from two <strong>on</strong>line SME directories; Metropolitan SME repository and Small Business<br />

directory. These two directories are diverse both in size, BEE status and ec<strong>on</strong>omic sector. The<br />

website “http://www.SMEportal.com/allcompanies” c<strong>on</strong>tains each SME’s product, company profile and<br />

supplies the website link. The sec<strong>on</strong>d source, Small Business directory<br />

(http://www.smallbusinessdirectory.co.za/) serves the same purpose but differs in categorising the<br />

SMEs by business sector instead of by name. Of the identified 300 SMEs <strong>on</strong>ly 30 had working<br />

websites sites. However there were some websites that had links disabled, these web sites were not<br />

c<strong>on</strong>sidered.<br />

5. Weighting<br />

To eliminate the emphasis <strong>on</strong> certain features the checklist features and functi<strong>on</strong>alities were not<br />

weighted. Elliot et al, (2000: 8) suggests that discussi<strong>on</strong>s about differing levels of importance for<br />

categories tends to distract attenti<strong>on</strong> from the main issue, this being the sophisticati<strong>on</strong> of websites to<br />

ascertain the intensity of eCommerce use of websites. If the element in the checklist was present ‘at<br />

an acceptable level’, then a single point is awarded. If the element is not present, no point is awarded.<br />

A more elaborate assessment could have been developed, like the use of a five-point Lickert scale for<br />

each element. However, there are inc<strong>on</strong>sistencies inherent in the approach and the variati<strong>on</strong> in the<br />

evaluati<strong>on</strong> framework reduces the reliability of the instrument (Elliot et al, 2000: 8).<br />

95


6. Findings and analysis<br />

6.1 Static c<strong>on</strong>tent<br />

David Freeme and Portia Gumede<br />

The frequency of static c<strong>on</strong>tent present in the surveyed SME websites is shown in Figure 1. The<br />

results show that all the sampled websites have product advertising. This supports the premise that<br />

SMEs websites are essentially digital brochures or catalogues. 97% of the sampled websites had<br />

graphics or images. Only 47% of the websites evaluated had pricing informati<strong>on</strong> for their products and<br />

/or services. More than half of the sites advertised without pricing informati<strong>on</strong>. These sites<br />

predominantly had enquiry or “request quote” email facilities to attain pricing informati<strong>on</strong>. Unexpected<br />

findings were a low 33% frequency of company history and high 81% frequency for other c<strong>on</strong>tacts<br />

informati<strong>on</strong>.<br />

Figure 1: Frequency of static c<strong>on</strong>tent in SME websites<br />

6.2 Transacti<strong>on</strong>al c<strong>on</strong>tent<br />

20 out of the 30 sampled websites had no transacti<strong>on</strong>al c<strong>on</strong>tent. The most comm<strong>on</strong>ly used feature in<br />

all the websites with transacti<strong>on</strong>al c<strong>on</strong>tent was an order form, to make either <strong>on</strong>line or offline orders.<br />

This feature is relatively easy to implement (Gwetu, 2009: 65) as opposed to the <strong>on</strong>line shopping<br />

features. This is particularly disc<strong>on</strong>certing as the majority of websites evaluated were for businesses<br />

that sold tangible products that could be traded <strong>on</strong>line. However, Mahdi and Steinmueller (2002: 11-<br />

12) state that the implementati<strong>on</strong> of transacti<strong>on</strong>s is a functi<strong>on</strong> that some SMEs choose to avoid<br />

regardless of the level of their experience (Mahdi and Steinmueller, 2002: 11-12).<br />

The low frequencies for transacti<strong>on</strong>al c<strong>on</strong>tent (even a 0% for the emerging mobile payment method:<br />

electr<strong>on</strong>ic wallet), suggest that SMEs avoid features that are difficult to implement and m<strong>on</strong>itor,<br />

possibly due to the security and backend obligati<strong>on</strong>s involved (Walcott, 2007: 265). SMEs, even<br />

though they are perceived largely entrepreneurial in a business c<strong>on</strong>text, portray a somewhat risk<br />

averse in their attitude towards the implementati<strong>on</strong> of complex features <strong>on</strong> their websites (Gwetu,<br />

2009: 65). In additi<strong>on</strong>, from the simplistic definiti<strong>on</strong> of eCommerce, which states that it is the<br />

purchasing and selling of products <strong>on</strong>line, the lack of transacti<strong>on</strong>al c<strong>on</strong>tent in SME websites in<br />

essence means they do not facilitate eCommerce (Yeung and Lu, 2002: 487). Figure 2 (below)<br />

illustrates the findings pertaining to transacti<strong>on</strong>al c<strong>on</strong>tent of the sampled websites.<br />

96


David Freeme and Portia Gumede<br />

Figure 2: Frequency of transacti<strong>on</strong>al c<strong>on</strong>tent in SME websites<br />

6.3 Interactive c<strong>on</strong>tent<br />

Figure 3 depicts the frequency of the interactive features and functi<strong>on</strong>alities used <strong>on</strong> the websites.<br />

The results show that 87% of the websites have email facilities or links for enquiries. The remaining<br />

13% either have no email c<strong>on</strong>tact or have <strong>on</strong>e in a ‘c<strong>on</strong>tact us’ secti<strong>on</strong>s. Of c<strong>on</strong>cern is that <strong>on</strong>ly 17%<br />

of the websites have links to social sites, even though this feature is easy to implement <strong>on</strong> a website<br />

(Gilmore, 2011: 1). The universal appeal and nature of social sites and their extended use as a<br />

marketing and feedback tool in the corporate envir<strong>on</strong>ment, this makes it an essential and inexpensive<br />

tool for the growth of SMEs (DiMicco et al, 2008: 712; Gilmore, 2011: 1).<br />

Figure 3: Frequency of interactive c<strong>on</strong>tent in SME websites<br />

97


David Freeme and Portia Gumede<br />

Aside from navigati<strong>on</strong>al features, such as links to distributors (57%) and QuickLinks (77%), all the<br />

websites evaluated scored below 30% <strong>on</strong> interactive features. Most of the websites <strong>on</strong>ly had email<br />

facilities and QuickLinks as interactive features. The use of audio and video features is extremely low<br />

<strong>on</strong> the surveyed websites. Search engines, be it Word or site search, was 30%. This low percentage<br />

could be indicative of the websites c<strong>on</strong>sisting of <strong>on</strong>ly a few pages of c<strong>on</strong>tent, thus reducing the<br />

justificati<strong>on</strong> for a search engine. These results reveal that currently SMEs put more emphasis <strong>on</strong><br />

informati<strong>on</strong>al c<strong>on</strong>tent as opposed to interactive features (Gwetu, 2009: 65).<br />

7. Maturity level of eCommerce based <strong>on</strong> results of the survey<br />

Currently, by most developing countries standards, South Africa has an advanced ICT infrastructure.<br />

This, however, even though it may have translated into an increase in adopti<strong>on</strong> of IT by businesses,<br />

does not translate into a high maturity level for eCommerce (Molla et al, 2006: 6; Atrostic, 1999: 96).<br />

The above survey results indicate that most of the websites host static c<strong>on</strong>tent implying that South<br />

African SMEs are at stage 2 maturity based <strong>on</strong> the stage models of McKay et al, (2000: 3), Ho (1997)<br />

and Burgess and Cooper (2000). These findings suggest that these SMEs are still experimenting with<br />

<strong>on</strong>line presence and this is reflected in the low maturity rating. Stockdale and Standing (2006: 386)<br />

term this phase “paddling, where SMEs are likely to have email and internet c<strong>on</strong>necti<strong>on</strong> but are<br />

hesitant to exploit these technologies to their full potential”.<br />

Additi<strong>on</strong>al findings are the sampled SME websites were predominantly composed of static c<strong>on</strong>tent.<br />

Two thirds of the sampled websites had no transacti<strong>on</strong>al c<strong>on</strong>tent and little interactive c<strong>on</strong>tent. The<br />

sophisticati<strong>on</strong> and quality of the websites was low. The percentage of SMEs possessing a website<br />

was low when c<strong>on</strong>sidering the total number of website links and SME searches (over 300 searches)<br />

that were performed in the process of applying the checklist. 70% of the surveyed SMEs had no<br />

transacti<strong>on</strong>al and/or interactive features at the same time. The fact that the SMEs surveyed host<br />

websites had low sophisticati<strong>on</strong> and hardly any quality c<strong>on</strong>tent reinforced the finding of a low<br />

eCommerce maturity level. The maturity level of eCommerce implemented by the SMEs in the South<br />

African retail and wholesale sector should be addressed as <strong>on</strong>ly a small percentage of SMEs have<br />

more than a simplistic web presence <strong>on</strong>line. In a study c<strong>on</strong>ducted in year 2000, <strong>on</strong>ly 1% of the<br />

sampled SMEs had achieved an integrated eCommerce status. This figure was predicted to increase<br />

to 36% by the end of 2004 (Molla and Heeks, 2007: 94). The findings of this research do not support<br />

this projecti<strong>on</strong>. The findings of this research support the premise that eCommerce is not an agenda<br />

item for most SA SMEs with websites.<br />

8. C<strong>on</strong>clusi<strong>on</strong><br />

The paper, and the research in general, has attempted to find a method of measuring the maturity<br />

levels of South African SMEs by using a checklist based <strong>on</strong> the stages of growth model. Initial<br />

indicati<strong>on</strong>s are that this method of research could be quite valuable as it overcomes several of the<br />

shortcomings of current h<strong>on</strong>ours degree academic research like the short time frame for h<strong>on</strong>ours<br />

degree research, n<strong>on</strong>-return of questi<strong>on</strong>naires, different interpretati<strong>on</strong>s of survey questi<strong>on</strong>s, etc.<br />

Further research and validati<strong>on</strong> <strong>on</strong> the derived checklist is currently being undertaken.<br />

References<br />

Ally, M., Cater-Steel, A. and Toleman, M. 2007. A web site sophisticati<strong>on</strong> model based <strong>on</strong> value-added<br />

technology soluti<strong>on</strong>s and services. ACIS 2007 Proceedings, December 5-7, Toowoomba, 99(1), 1008-1016.<br />

Atrostic, B. 1999. Defining and measuring e-Commerce: a status report. The Brookings Instituti<strong>on</strong>, 4(99), 1-19.<br />

Bauer, C. and Scharl, A. 2000. Quantitive evaluati<strong>on</strong> of web site c<strong>on</strong>tent and structure. Internet Research:<br />

Electr<strong>on</strong>ic Networking Applicati<strong>on</strong>s and Policy. 10 (1): 31-43.<br />

Beck, R., Wigand, R. and K<strong>on</strong>ig, W. 2005. The diffusi<strong>on</strong> and efficient use of electr<strong>on</strong>ic commerce am<strong>on</strong>g small<br />

and medium sized enterprises: an internati<strong>on</strong>al three industry survey. Electr<strong>on</strong>ic Markets, 15 (1), 38-52.<br />

Berry, A., v<strong>on</strong> Blottnitz, M., Cassim, R., Kesper, A., Rajaratnam, B. and van Seventer, D. 2002. The ec<strong>on</strong>omics of<br />

SMEs in South Africa. Trade and Industrial Policy Strategies, December (2002), 1-116.<br />

Burgess, L. and Cooper, Extending the Viability of MICA (Model of Internet Commerce Adopti<strong>on</strong>) as a Metric for<br />

Explaining the Process of Business Adopti<strong>on</strong> of Internet Commerce. Internati<strong>on</strong>al <str<strong>on</strong>g>C<strong>on</strong>ference</str<strong>on</strong>g> <strong>on</strong><br />

Telecommunicati<strong>on</strong>s and Electr<strong>on</strong>ic Commerce, Dallas (November).<br />

Burgess, L., Cooper, J., Gibb<strong>on</strong>s-Parrish, B. and Alcock, C. 2009. A l<strong>on</strong>gitudinal study of the web by regi<strong>on</strong>al<br />

tourism organisati<strong>on</strong>s (RTOs) in Australia. 22nd Bled e<str<strong>on</strong>g>C<strong>on</strong>ference</str<strong>on</strong>g> Proceedings, June 15-22, Slovenia, 519-<br />

531.<br />

Chang, L., Kirk, P. and Litecky, C. 2001. Design quality of websites for electr<strong>on</strong>ic commerce: Fortune 1000<br />

webmasters’ evaluati<strong>on</strong>s. Electr<strong>on</strong>ic Markets, 10(2), 120-129.<br />

98


David Freeme and Portia Gumede<br />

Cloete, E.2002. SME’s in South Africa: Acceptance and Adopti<strong>on</strong> of e-Commerce. Department of Informati<strong>on</strong><br />

Systems, University of Cape Town, 1-12.<br />

Daniel, E., Wils<strong>on</strong>, H. and Myers, A. 2002. Adopti<strong>on</strong> of ecommerce by SMEs in the UK: towards a stage model.<br />

Internati<strong>on</strong>al Small Business Journal, 20(3), 253-270.<br />

DiMicco, J., Millen, D., Geyer, W., Dugan, C., Brownholtz, B. and Muller, M. 2008. Motivati<strong>on</strong>s for social<br />

networking at work. ACM, 8(2008), 711-720.<br />

Doolin, B., Burgess, L. and Cooper, J. 2002. Evaluating the use of the web for tourism marketing: a case study<br />

from New Zealand. Tourism <strong>Management</strong>, 23 (2002), 557-561.<br />

Elliot, R., Morup-Petersen, A. and Bjorn-Anders<strong>on</strong>. 2000. Towards a framework for evaluati<strong>on</strong> of commercial web<br />

sites. Electr<strong>on</strong>ic Commerce: The end of the beginning. Proceedings of the 13 th internati<strong>on</strong>al electr<strong>on</strong>ic<br />

commerce, June 19-21, Slovenia, 1-15.<br />

Fisher, J., Craig, A. and Bentley, J. 2007. Moving from a web presence to e-Commerce: The importance of a<br />

business-web strategy for small-business owners. Electr<strong>on</strong>ic Markets, 17(4): 253 -262.<br />

Garcia-Borbolla, A., Larran, M. and Lopez, R. 2005. Empirical evidence c<strong>on</strong>cerning SMMEs’ corporate websites:<br />

explaining factors, strategies and reporting. The Internati<strong>on</strong>al Journal of Digital Accounting Research, 5(10),<br />

171-202.<br />

Green, D. and Pears<strong>on</strong>, M. 2010. Integrating website usability with the electr<strong>on</strong>ic<br />

commerce acceptance model. Behaviour & Informati<strong>on</strong> Technology, 30(2):181-199.<br />

Gilmore, J. 2011. How to Put a Facebook Link <strong>on</strong> My Website. [Online]. Available: How to Put a Facebook Link<br />

<strong>on</strong> My Website | eHow.com http://www.ehow.com/how_5832801_put-facebookwebsite.html#ixzz1ZnyFyRAU<br />

[Accessed: 01 October 2011].<br />

Green, D. and Pears<strong>on</strong>, M. 2010. Integrating website usability with the electr<strong>on</strong>ic<br />

Commerce acceptance model. Behaviour and Informati<strong>on</strong> Technology, 30(2):181-199.<br />

Gwetu, M.V. 2009. Web applicati<strong>on</strong> by South African health instituti<strong>on</strong>s. University of Venda, 60-61.Lightner, N.J.<br />

2004. Evaluating e-commerce functi<strong>on</strong>ality with a focus <strong>on</strong> customer service. Communicati<strong>on</strong>s of the ACM,<br />

47(10), 88-92.<br />

Ho, J. 1997. Evaluating the World Wide Web: A Global Study of Commercial Sites. Journal of Computer-<br />

Mediated Communicati<strong>on</strong>, vol. 3 no. 1.<br />

Lighter, N.J. 2004, Evaluating e-commerce functi<strong>on</strong>ality with a focus <strong>on</strong> customer service. Communicati<strong>on</strong>s of the<br />

ACM, 47(10), 88-92.<br />

Mahdi, S. and Steinmueller, E. 2002. E-COMMERCE INDICATORS for the WWW: methodological approaches<br />

for assessing sectorial e-Commerce maturity. New Indicators for the Knowledge-Based Ec<strong>on</strong>omy, July, 1-<br />

69.<br />

McKay, J., Prananto, A. and P. Marshall. 2000. E-Business Maturity: The SOG-e Model. In Proceedings of the<br />

11th Australasian <str<strong>on</strong>g>C<strong>on</strong>ference</str<strong>on</strong>g> <strong>on</strong> Informati<strong>on</strong> Systems (ACIS). Queensland University of Technology,<br />

Brisbane, Australia.<br />

Mohamad, R. and Ismail, N.A. 2009. Electr<strong>on</strong>ic Commerce Adopti<strong>on</strong> in SME: The trend of Prior studies. Journal<br />

of Internet Banking and Commerce, 14(2), 1-16<br />

Molla, A. and Heeks, R. 2007. Exploring E-commerce benefits for businesses in a developing country. The<br />

Informati<strong>on</strong> Society, 23 (2): 95-108.<br />

Motjolopane, I.M. and Warden, S.C. 2007. Electr<strong>on</strong>ic commerce adopti<strong>on</strong> by SMEs Western Cape, South Africa.<br />

Informati<strong>on</strong> Resource <strong>Management</strong> Associati<strong>on</strong> <str<strong>on</strong>g>C<strong>on</strong>ference</str<strong>on</strong>g>, Vancouver: 1-15.<br />

Stockdale, R. and Standing, C., a. (2006) Classificati<strong>on</strong> Model to Support E-Commerce Adopti<strong>on</strong> Initiatives,<br />

Journal of Small Business Enterprise and Development, Vol, 13(3), 381-394.<br />

Rao, S.S., Metts, G. and M<strong>on</strong>ge, C.A. 2003. Electr<strong>on</strong>ic commerce development in small and medium sized<br />

enterprises: a stage model and its implicati<strong>on</strong>s. Business Process <strong>Management</strong> Journal, 9(1), 11-32.<br />

United Nati<strong>on</strong>s. 2002. Benchmarking E-government: A Global Perspective -Assessing the UN Member States.<br />

United Nati<strong>on</strong>s Divisi<strong>on</strong> for Public Ec<strong>on</strong>omics and Public Administrati<strong>on</strong>, 1-74<br />

Walcott, P.A. 2007. Evaluating the readiness of e-commerce websites. Internati<strong>on</strong>al Journal of Computers, 1(4),<br />

263-268.<br />

Wu, M., Zhang, L., Xing, Q., Dai, L. and Du, H. 2007. E-commerce adopti<strong>on</strong> in China’s service SMEs: a study<br />

from web usability perspective. Journal of Business Systems, Governance and Ethics, 2 (4), 1-13.<br />

Yeung, W. and Lu, M. 2002. Functi<strong>on</strong>al characteristics of commercial web sites: a l<strong>on</strong>gitudinal study in H<strong>on</strong>g<br />

K<strong>on</strong>g. Informati<strong>on</strong> and <strong>Management</strong>, 41 (2004): 483-495<br />

99


Advancing GeoMarketing Analyses with Improved Spatiotemporal<br />

Distributi<strong>on</strong> of Populati<strong>on</strong> at High Resoluti<strong>on</strong><br />

Sérgio Freire and Teresa Santos<br />

e-GEO Research Centre for Geography and Regi<strong>on</strong>al Planning, FCSH,<br />

Universidade Nova de Lisboa, Lisboa, Portugal<br />

sfreire@fcsh.unl.pt<br />

teresasantos@fcsh.unl.pt<br />

Abstract: Knowing the spatiotemporal distributi<strong>on</strong> of populati<strong>on</strong> at the local level is fundamental for many<br />

applicati<strong>on</strong>s, including risk management, health and envir<strong>on</strong>mental studies, territorial planning and management,<br />

and GeoMarketing. Census figures register where people reside and usually sleep, and are frequently the <strong>on</strong>ly<br />

data source available for such analyses. Currently, the analysis of service areas and populati<strong>on</strong> served is mostly<br />

made c<strong>on</strong>sidering <strong>on</strong>ly census data as source of populati<strong>on</strong> distributi<strong>on</strong>, while some businesses clearly serve<br />

mostly a daytime populati<strong>on</strong>. However, populati<strong>on</strong> density is not c<strong>on</strong>stant within census enumerati<strong>on</strong> areas. Also,<br />

due to human activities, populati<strong>on</strong> counts and their distributi<strong>on</strong> vary widely from nighttime to daytime, especially<br />

in metropolitan areas, and this variati<strong>on</strong> is not captured by census data. Raster dasymetric mapping within<br />

geographic modeling allows transforming raw populati<strong>on</strong> counts to populati<strong>on</strong> density limited to specific areas<br />

where the variable is present, in more detailed temporal periods, by using ancillary data sets and z<strong>on</strong>al<br />

interpolati<strong>on</strong>. In GeoMarketing, this informati<strong>on</strong> is especially useful for retail sales, banking, insurance, lodging,<br />

real estate, and franchising. These refined distributi<strong>on</strong>s can be used to improve such analyses as site selecti<strong>on</strong>,<br />

service area and populati<strong>on</strong> served, assessment of potential markets, routing activities, locati<strong>on</strong>-allocati<strong>on</strong>, and<br />

gravity models. This study uses such a dasymetric mapping approach for detailed modeling and mapping of the<br />

spatiotemporal distributi<strong>on</strong> of populati<strong>on</strong> in the daily cycle. These data sets are used to assess the locati<strong>on</strong> and<br />

the varying populati<strong>on</strong> c<strong>on</strong>tained in the service areas of existing and prospective commercial facilities in the daily<br />

cycle, for different types of businesses. Applicati<strong>on</strong>s in GeoMarketing using spatial analysis are illustrated for<br />

three different scenarios involving private sector services where maximizing coverage of target populati<strong>on</strong> is<br />

paramount for success. The case studies show that when the spatiotemporal distributi<strong>on</strong> of populati<strong>on</strong> is<br />

c<strong>on</strong>sidered, the obtained set of soluti<strong>on</strong>s differs from the <strong>on</strong>e produced by using census-based data al<strong>on</strong>e. The<br />

results dem<strong>on</strong>strate that enhancing populati<strong>on</strong> distributi<strong>on</strong> data through geographical modeling can greatly<br />

benefit spatial analysis in GeoMarketing, resulting in the producti<strong>on</strong> of better informati<strong>on</strong> that ultimately allows<br />

improved decisi<strong>on</strong>-making.<br />

Keywords: GeoMarketing, populati<strong>on</strong> distributi<strong>on</strong>, dasymetric mapping, service area, maximum coverage,<br />

Oeiras<br />

1. Introducti<strong>on</strong><br />

1.1 GeoMarketing as a structured process<br />

GeoMarketing is a fairly recent discipline that combines the power of geographic visualizati<strong>on</strong> and<br />

analysis with Marketing techniques and insight, aiming at more efficiently attaining the ultimate goal of<br />

the latter: to sell products, services, or ideas. Although the c<strong>on</strong>cept of marketing mix has l<strong>on</strong>g<br />

included ‘Place’ am<strong>on</strong>g the four ‘Ps’ (others being Product, Price, and Promoti<strong>on</strong>) (McCarthy 1960),<br />

its importance has been underestimated in formal technical analyses that support decisi<strong>on</strong>-making.<br />

Driving the need for GeoMarketing is the basic premise that markets vary from place to place (and<br />

with time) and that business strategies should take this fact into account. For many businesses, the<br />

decisi<strong>on</strong> of where to locate their commercial outlets will be the most important determinant of their<br />

success.<br />

The emergence of GeoMarketing was facilitated by advancements in the spatial analysis and<br />

visualizati<strong>on</strong> capabilities of Geographic Informati<strong>on</strong> Systems (GIS), but their usefulness for c<strong>on</strong>ducting<br />

studies in business and ec<strong>on</strong>omics remains to be fully explored (Cheng et al. 2007; Mishra 2009). As<br />

a structured process supporting decisi<strong>on</strong>-making, GeoMarketing analyses are more than a simple<br />

task, instead involving the following sequential steps: (1) formulating the problem, (2) obtaining and<br />

processing the required data, (3) c<strong>on</strong>ducting the analysis, and (4) producing c<strong>on</strong>clusi<strong>on</strong>s and<br />

recommendati<strong>on</strong>s. Spatial and n<strong>on</strong>-spatial data sets are required to characterize both the supply<br />

(facilities, service, product, competitors) and the demand (populati<strong>on</strong>, existing and potential<br />

customers) in a given market. However, as with every informati<strong>on</strong> system, the quality of the results is<br />

never higher than the accuracy of the input data used.<br />

100


Sérgio Freire and Teresa Santos<br />

C<strong>on</strong>cerning their geographical scale or scope, analyses can be c<strong>on</strong>ducted at four levels: (i)<br />

c<strong>on</strong>tinental, (ii) nati<strong>on</strong>al (am<strong>on</strong>g countries), (iii) regi<strong>on</strong>al (comparing regi<strong>on</strong>s or cities), or (iv) local<br />

(within a city or settlement). Regarding the required socioec<strong>on</strong>omic data (including demographic) to<br />

match these levels of analyses, (i) and (ii) require country-level totals, (iii) demands data by<br />

municipalities or communes, while (iv) requires disaggregated data that represent intra-urban<br />

variati<strong>on</strong>s.<br />

1.2 The relevance of populati<strong>on</strong> distributi<strong>on</strong> for GeoMarketing<br />

Knowing the spatiotemporal distributi<strong>on</strong> of populati<strong>on</strong> at the local level is fundamental for many<br />

applicati<strong>on</strong>s, including risk management, health and envir<strong>on</strong>mental studies, territorial planning and<br />

management, and also GeoMarketing (Freire 2010). The capability of obtaining accurate simple<br />

populati<strong>on</strong> totals within the service area is often a basic indicator of potential demand for a service<br />

and corresp<strong>on</strong>ding financial success, especially for those which have a local demand and serve the<br />

immediate neighborhood.<br />

A typical problem for GeoMarketing has been how to best locate a commercial facility in order to<br />

maximize the potential customer coverage and therefore profit (Je<strong>on</strong>g et al. 2008). C<strong>on</strong>ducting such<br />

an analysis requires as a minimum data <strong>on</strong> populati<strong>on</strong>/potential customer distributi<strong>on</strong> and informati<strong>on</strong><br />

c<strong>on</strong>cerning the business. However, there is a surprising lack of academic research studying the<br />

importance of demographic data for such analyses and testing more advanced data sources bey<strong>on</strong>d<br />

those solely based <strong>on</strong> the census.<br />

Census figures register where people reside and usually sleep, and are frequently the <strong>on</strong>ly data<br />

source available for such analyses. Currently, the analysis of service areas and populati<strong>on</strong> served in<br />

Portugal and many countries is made c<strong>on</strong>sidering <strong>on</strong>ly residence-based (nighttime) census data as<br />

source of populati<strong>on</strong> distributi<strong>on</strong>, while some businesses clearly serve mostly a daytime populati<strong>on</strong>.<br />

However, populati<strong>on</strong> density is not c<strong>on</strong>stant within census enumerati<strong>on</strong> areas, although it is<br />

comm<strong>on</strong>ly represented as such. Also, due to human activities, populati<strong>on</strong> counts and their distributi<strong>on</strong><br />

vary widely from nighttime to daytime, especially in metropolitan areas, and may be misrepresented<br />

by census data.<br />

Geographic modeling and dasymetric mapping allows re-distributing populati<strong>on</strong> to specific areas<br />

where it is present in more detailed temporal periods, by using ancillary data and z<strong>on</strong>al interpolati<strong>on</strong><br />

(Eicher and Brewer 2001). In GeoMarketing, this informati<strong>on</strong> is especially useful for retail sales,<br />

banking, insurance, lodging, real estate, and franchising. These refined distributi<strong>on</strong>s can be used to<br />

improve such analyses as site selecti<strong>on</strong>, service area and populati<strong>on</strong> served, assessment of potential<br />

markets, routing activities, locati<strong>on</strong>-allocati<strong>on</strong>, and gravity models.<br />

The present work aims at presenting the development of a dasymetric mapping approach for detailed<br />

modeling and mapping of the spatiotemporal distributi<strong>on</strong> of populati<strong>on</strong> in the daily cycle, and<br />

dem<strong>on</strong>strating its value for improving geographical analyses in GeoMarketing. Applicati<strong>on</strong>s using<br />

spatial analysis are illustrated for three different scenarios involving private sector services where<br />

maximizing populati<strong>on</strong> served (i.e. potential demand) is paramount for success.<br />

2. Study area and data<br />

2.1 Study area<br />

Detailed spatiotemporal populati<strong>on</strong> modeling was performed for Oeiras and Cascais, two of the<br />

eighteen municipalities that comprise the Lisb<strong>on</strong> Metropolitan Area (LMA), the main metropolitan area<br />

in Portugal. Dem<strong>on</strong>strati<strong>on</strong>s of applicati<strong>on</strong>s of these data sets in Geomarketing analyses are<br />

implemented in the municipality of Oeiras (Figure 1). This municipality occupies 46 km 2 and has a<br />

resident populati<strong>on</strong> of 162,128. Density of resident populati<strong>on</strong> varies significantly throughout the study<br />

area, from high density in multi-story apartments to low-density in rural areas. Even at the census<br />

block group level, some polyg<strong>on</strong>s are enormous and do not reflect their uneven populati<strong>on</strong> density.<br />

Despite the ‘gravitati<strong>on</strong>al pull’ of the adjacent city of Lisb<strong>on</strong> (the nati<strong>on</strong>al capital), Oeiras recently<br />

created several technological and office parks and has acquired an intensive tertiary activity.<br />

Therefore daytime populati<strong>on</strong> displays distinct spatial distributi<strong>on</strong> and densities from the census,<br />

totaling 148,937 people.<br />

101


Sérgio Freire and Teresa Santos<br />

This area has ideal characteristics for this study, namely with regard to urban and suburban character<br />

and presence of str<strong>on</strong>g ec<strong>on</strong>omic activity.<br />

Figure 1: Locati<strong>on</strong> of the study area in Portugal and in the Lisb<strong>on</strong> Metropolitan Area<br />

2.2 Data<br />

Input variables used for modeling populati<strong>on</strong> distributi<strong>on</strong> include both physiographic and statistical<br />

data. In the first group are census tracts, street centerlines and land use and land cover (LULC), while<br />

the sec<strong>on</strong>d includes census counts (INE 2001), data <strong>on</strong> workforce by workplaces, and commuting<br />

statistics (INE 2003) for the study area. These data were obtained from various sources and in<br />

different formats which are listed in Table 1. The target year for modeling populati<strong>on</strong> density is 2001.<br />

Table 1: Main input datasets used for modeling nighttime and daytime populati<strong>on</strong><br />

Data set Source Date Data type<br />

Street centerlines Private 2004 Vector polyline<br />

LULC (COS90; CLC2000) Public 1990; 2000 Vector polyg<strong>on</strong><br />

Census block groups Public 2001 Vector polyg<strong>on</strong><br />

Census statistics Public 2001 Database (MS Access)<br />

Workplaces and employment Public 2001 Table<br />

Commuting statistics Public 2001 Table (O/D matrix)<br />

COS90 is a digital LULC map at the scale 1:25,000 covering almost the entire country, however it<br />

dates from 1990. Therefore, to ensure temporal c<strong>on</strong>sistency am<strong>on</strong>g input data sets, it was decided to<br />

update it to some extent using the more recent CORINE Land Cover database for the year 2000.<br />

3. Geographic modeling and analysis<br />

The methodology was implemented in a Geographic Informati<strong>on</strong> System (GIS) and includes two main<br />

stages: a) modeling spatiotemporal populati<strong>on</strong> distributi<strong>on</strong> and b) building and analyzing sample<br />

Geomarketing applicati<strong>on</strong>s.<br />

3.1 Modeling spatiotemporal populati<strong>on</strong> distributi<strong>on</strong>s<br />

The modeling of populati<strong>on</strong> distributi<strong>on</strong> is based <strong>on</strong> dasymetric mapping using street centerlines as<br />

spatial reference unit to allocate populati<strong>on</strong> counts. The most recent statistical and census data<br />

(2001) provide the populati<strong>on</strong> counts for each daily period, while physiographic data sets define the<br />

spatial units (i.e., grid cells) used to disaggregate those counts. This general approach was proposed<br />

by the Los Alamos Nati<strong>on</strong>al Laboratory (New Mexico, USA) to map daytime and nighttime populati<strong>on</strong><br />

distributi<strong>on</strong>s in the US at 250-m resoluti<strong>on</strong> (McPhers<strong>on</strong> and Brown 2003), and is adapted and applied<br />

to Portugal.<br />

102


Sérgio Freire and Teresa Santos<br />

The map of nighttime populati<strong>on</strong> distributi<strong>on</strong> was obtained by using a grid binary dasymetric mapping<br />

method to disaggregate residential populati<strong>on</strong> from census z<strong>on</strong>es to residential streets. First,<br />

available digital LULC maps were improved, relevant classes selected, and combined, in order to<br />

identify residential land use. Street centerlines were also modified in order to better represent the road<br />

network existing in 2001. Then, freeways are removed from c<strong>on</strong>siderati<strong>on</strong> and the resulting eligible<br />

streets are combined with residential land use classes from LULC data to obtain residential streets.<br />

These are subsequently rasterized at 25 m and the populati<strong>on</strong> from census block groups (source<br />

z<strong>on</strong>es) are interpolated to the respective residential street cells (target z<strong>on</strong>es) using areal weighting<br />

The modeling of daytime populati<strong>on</strong> distributi<strong>on</strong> c<strong>on</strong>siders mobility statistics. It results from the<br />

combinati<strong>on</strong> of two comp<strong>on</strong>ents: a) the daytime populati<strong>on</strong> in their places of work or study – the<br />

workforce populati<strong>on</strong> surface, and b) the populati<strong>on</strong> that remains home during the day - the daytime<br />

residential populati<strong>on</strong> grid. The latter is obtained by multiplying the nighttime distributi<strong>on</strong> by the<br />

percentage of resident populati<strong>on</strong> who, according to official statistics (INE 2003), does not commute<br />

to work or school. In the absence of other informati<strong>on</strong>, it is assumed that n<strong>on</strong>-commuters remain in<br />

their residences in the daytime period. The workforce populati<strong>on</strong> surface was created by<br />

georeferencing 2167 workplaces and schools and respective workforce and students in the study<br />

area. 562 of these facilities were georeferenced manually using ancillary data and field work. The<br />

remainder workplaces were geocoded to the street centerlines using their addresses.<br />

Using this methodology, four raster populati<strong>on</strong> distributi<strong>on</strong> surfaces were produced, at 25 m<br />

resoluti<strong>on</strong>: (i) nighttime (residential) populati<strong>on</strong> (Figure 2), (ii) daytime residential populati<strong>on</strong>, (iii)<br />

daytime worker and student populati<strong>on</strong>, and (iv) total daytime populati<strong>on</strong> (Figure 3). Additi<strong>on</strong>ally, an<br />

ambient populati<strong>on</strong> surface is produced by computing a weighted average of nighttime and daytime<br />

distributi<strong>on</strong>s, c<strong>on</strong>sidering the proporti<strong>on</strong> of nighttime and daytime periods occurring in a typical 7-day<br />

weekly cycle.<br />

Figure 2: Grid of nighttime populati<strong>on</strong> distributi<strong>on</strong> in Oeiras in 3D<br />

The resulting 25-m populati<strong>on</strong> grids were subsequently aggregated to 50 m cells for analysis and<br />

visualizati<strong>on</strong> purposes, thus representing densities by 2,500 m 2 (0.25 ha). The nighttime distributi<strong>on</strong><br />

was validated using the higher-resoluti<strong>on</strong> census blocks units as reference (i.e. ground truth) in a<br />

correlati<strong>on</strong> analysis. A correlati<strong>on</strong> coefficient (Pears<strong>on</strong>’s r) of 0.79 was obtained, showing a good<br />

performance of the model. Additi<strong>on</strong>al details <strong>on</strong> the populati<strong>on</strong> distributi<strong>on</strong> modeling are provided in<br />

Freire (2010).<br />

This method also efficiently accommodates people that work at home, by not assuming that all active<br />

populati<strong>on</strong> leaves their residences during the workday. The main value of these results includes the<br />

increased spatial resoluti<strong>on</strong> of nighttime distributi<strong>on</strong> (higher than census data), the fact that both<br />

103


Sérgio Freire and Teresa Santos<br />

nighttime and daytime distributi<strong>on</strong>s share the same spatial reference basis (therefore support cell-bycell<br />

comparis<strong>on</strong>, and that previously unavailable daytime distributi<strong>on</strong> is represented.<br />

Figure 3: Grid of daytime populati<strong>on</strong> distributi<strong>on</strong> in Oeiras in 3D<br />

3.2 Sample applicati<strong>on</strong>s to GeoMarketing<br />

For dem<strong>on</strong>strating the usefulness of detailed spatiotemporal populati<strong>on</strong> in GeoMarketing, three typical<br />

sample applicati<strong>on</strong>s using demographic analysis are illustrated in the municipality of Oeiras. One<br />

applicati<strong>on</strong> c<strong>on</strong>cerns quantifying the actual coverage for an existing network of facilities, whereas the<br />

other two applicati<strong>on</strong>s c<strong>on</strong>cern expansi<strong>on</strong> and locati<strong>on</strong> planning, aiming at selecting the optimum<br />

locati<strong>on</strong> (in practice the best) for new facilities, c<strong>on</strong>sidering both their target demographics and hours<br />

of operati<strong>on</strong>.<br />

The scenarios involve types of business services having local influence and markets, therefore the<br />

service areas are based <strong>on</strong> accessibility over the street network, computed c<strong>on</strong>sidering either metric<br />

distance or time. Although the business data are fictitious, the remaining data analyses are real and<br />

accurate.<br />

3.2.1 Case study A: Assessing populati<strong>on</strong> served by existing facilities network<br />

A bank wants to quantify the current populati<strong>on</strong> coverage for its existing network of four branches in<br />

the municipality of Oeiras (Figure 4). Accessibility is measured in metric distance and a 1-km service<br />

area is defined for each facility, using the road network.<br />

Figure 5 illustrates assessment of served populati<strong>on</strong> using the nighttime distributi<strong>on</strong> (A) and the<br />

daytime distributi<strong>on</strong> (B).<br />

Results of the assessment of served populati<strong>on</strong> within the service areas (Table 2) show a quite<br />

different outcome obtained with each of the two surfaces in terms of raw figures and ranking of<br />

facilities. While based <strong>on</strong> the census-like nighttime distributi<strong>on</strong>, facility #3 serves the most people<br />

(4372) and facility #4 the least, using the populati<strong>on</strong> surface that matches banks’ daytime operating<br />

hours it is facility #1 which serves the most people (8618) and facility #3 the least. Overall, 19173<br />

people are served by the four facilities c<strong>on</strong>sidering their daytime distributi<strong>on</strong>, compared to 13073<br />

when the assessment uses the residential distributi<strong>on</strong> al<strong>on</strong>e.<br />

3.2.2 Case study B: Site selecti<strong>on</strong> for daytime service<br />

A restaurant chain operates two profitable facilities and wishes to open a third facility in the vicinity.<br />

Since this business targets mostly a local market due to their quick service and affordable meals, it<br />

makes sense to measure accessibility as time and define its core service area as being within five<br />

104


Sérgio Freire and Teresa Santos<br />

minutes walking distance (Figure 6). Am<strong>on</strong>g the two locati<strong>on</strong>s available for expansi<strong>on</strong> (#1 and #2),<br />

which <strong>on</strong>e has the most populati<strong>on</strong> and potential within its service area?<br />

Figure 4: Case study A: locati<strong>on</strong>s of existing banking facilities<br />

Figure 5: Potential customers assessed using the nighttime distributi<strong>on</strong> (A) and the daytime worker<br />

and student populati<strong>on</strong> (B)<br />

Table 2: Populati<strong>on</strong> served and ranking of banking facilities using the nighttime and daytime<br />

populati<strong>on</strong><br />

Facility #<br />

Nighttime<br />

Populati<strong>on</strong> Rank Populati<strong>on</strong><br />

Daytime<br />

Rank<br />

1 4250 2 8618 1<br />

2 3714 3 3220 3<br />

3 4372 1 3202 4<br />

4 737 4 4133 2<br />

Total 13073 -- 19173 --<br />

105


Sérgio Freire and Teresa Santos<br />

Figure 6: Case study B: existing (#3, #4) and prospective locati<strong>on</strong>s (#1, #2) and their service areas<br />

Figure 7 shows a comparis<strong>on</strong> of potential customers for the prospective locati<strong>on</strong>s assessed using the<br />

nighttime distributi<strong>on</strong> (A) and the daytime worker and student populati<strong>on</strong> (B).<br />

Figure 7: Potential customers assessed using the nighttime distributi<strong>on</strong> (A) and the daytime worker<br />

and student populati<strong>on</strong> (B) grids<br />

Whereas site #1 serves the most people in nighttime (1748), these are local residents less likely to be<br />

regular customers. Using the daytime worker and student populati<strong>on</strong> for analysis (i.e. the target<br />

demographics) shows that instead site #2 serves the most potential customers (1900 vs. 273).<br />

3.2.3 Case study C: Site selecti<strong>on</strong> for daytime and nighttime business<br />

Am<strong>on</strong>g three potential locati<strong>on</strong>s, a movie rental chain wants to select <strong>on</strong>e site for expansi<strong>on</strong>. Since<br />

this a proximity-based business, accessibility can be measured as time and the service area defined<br />

by a five-minute walking distance (Figure 8). Extending its operating hours late into the night, this<br />

business targets both displaced (workers and students) and residential customers, in the daytime and<br />

nighttime periods.<br />

Figure 8: Case study C: prospective locati<strong>on</strong>s and respective service areas<br />

106


Sérgio Freire and Teresa Santos<br />

Figure 9 shows a comparis<strong>on</strong> of potential customers within their service areas using the nighttime<br />

distributi<strong>on</strong> (A) and the ambient populati<strong>on</strong> (B) surfaces.<br />

Figure 9: Potential customers assessed using the nighttime (A) and the ambient populati<strong>on</strong> (B) grids<br />

The analysis shows that locati<strong>on</strong> #3 would serve the most people in nighttime (1170). However, when<br />

the daytime populati<strong>on</strong> is also c<strong>on</strong>sidered (via the ambient populati<strong>on</strong> grid) in order to account for the<br />

daytime and nighttime hours of operati<strong>on</strong>, locati<strong>on</strong> #1 emerges as having the greatest potential.<br />

These case studies dem<strong>on</strong>strate that GeoMarketing analyses relying solely <strong>on</strong> census-based data<br />

(nighttime) for characterizati<strong>on</strong> of populati<strong>on</strong> served and business potential could misestimate the<br />

prospective customer base c<strong>on</strong>tained in the service areas and indicate a soluti<strong>on</strong> with less potential<br />

for success.<br />

4. C<strong>on</strong>clusi<strong>on</strong>s<br />

In metropolitan areas, the populati<strong>on</strong> distributi<strong>on</strong> and densities are not static, varying significantly in<br />

the daily cycle. Knowing and using the spatiotemporal distributi<strong>on</strong> of populati<strong>on</strong> at the local level can<br />

greatly increase the quality of basic spatial analyses in GeoMarketing. An approach was presented<br />

that allows combining existing geographic informati<strong>on</strong> with official statistics to model and map<br />

nighttime and daytime populati<strong>on</strong> distributi<strong>on</strong> at high spatial resoluti<strong>on</strong>, able to support local-level<br />

analysis. The model integrates locati<strong>on</strong> of workplaces and schools with daily commuting statistics.<br />

The segmentati<strong>on</strong> of daytime distributi<strong>on</strong> into residential and worker and student grids further benefits<br />

GeoMarketing analyses. Three sample applicati<strong>on</strong>s of these populati<strong>on</strong> data sets were presented,<br />

c<strong>on</strong>sidering the nature of the business activity, their target demographics and hours of operati<strong>on</strong>.<br />

Results show that using populati<strong>on</strong> distributi<strong>on</strong> data disaggregated in space and time can significantly<br />

increase the detail and accuracy of spatial analysis, having the potential to greatly improve studies in<br />

GeoMarketing.<br />

Future developments should focus <strong>on</strong>: better modeling of ‘distributed activities’ (e.g., cleaning,<br />

security), and accounting for people present in transportati<strong>on</strong> networks or involved in leisure and<br />

shopping activities; increased temporal segmentati<strong>on</strong>s of populati<strong>on</strong> distributi<strong>on</strong>, so as to represent<br />

differences <strong>on</strong> a weekly basis (workdays vs. week-end) or <strong>on</strong> a seas<strong>on</strong>al basis (winter vs. summer);<br />

and the use of statistical sources bey<strong>on</strong>d census demographics to c<strong>on</strong>sider tourism influx in areas<br />

and periods where that activity is important.<br />

Acknowledgements<br />

The authors thank GeoPoint Lda. (www.geopoint.pt) whose kind support motivated the initial<br />

development of this work.<br />

References<br />

Cheng, E.W.L., Li, H., Yu. L. (2007) “A GIS Approach to Shopping Mall Locati<strong>on</strong> Selecti<strong>on</strong>”. Building and<br />

Envir<strong>on</strong>ment, Vol. 42, No. 2, pp 884-892.<br />

Eicher, C.L. and Brewer, C.A. (2001) “Dasymetric Mapping and Areal Interpolati<strong>on</strong>: Implementati<strong>on</strong> and<br />

Evaluati<strong>on</strong>”. Cartography and Geographic Informati<strong>on</strong> Science, Vol. 28, pp125–138.<br />

Freire, S. 2010 “Modeling of Spatiotemporal Distributi<strong>on</strong> of Urban Populati<strong>on</strong> at High Resoluti<strong>on</strong> - Value for Risk<br />

Assessment and Emergency <strong>Management</strong>”. In K<strong>on</strong>ecny, M., Zlatanova, S. & Bandrova, T.L. (eds.),<br />

107


Sérgio Freire and Teresa Santos<br />

Geographic Informati<strong>on</strong> and Cartography for Risk and Crisis <strong>Management</strong>, Lecture Notes in<br />

Geoinformati<strong>on</strong> and Cartography. Springer Berlin Heidelberg, pp 53-67.<br />

INE (Instituto Naci<strong>on</strong>al de Estatística), (2001) “Recenseamento Geral da População e da Habitação”. INE Lisboa.<br />

INE (Instituto Naci<strong>on</strong>al de Estatística), (2003) “Movimentos Pendulares e Organização do Território<br />

Metropolitano: Área Metropolitana de Lisboa e Área Metropolitana do Porto 1991-2001”. INE, Lisboa.<br />

Je<strong>on</strong>g, H., Um, J., S<strong>on</strong>, S.-W., Lee, S.-I., Kim, B.-J. (2008) “Finding Optimal Positi<strong>on</strong> of Facilities based <strong>on</strong><br />

Populati<strong>on</strong> Density”, Proceedings of Internati<strong>on</strong>al Workshop and <str<strong>on</strong>g>C<strong>on</strong>ference</str<strong>on</strong>g> <strong>on</strong> Network Science, Norwich,<br />

UK, June 23-27.<br />

McCarthy, E.J. (1960) Basic Marketing: A Managerial Approach, Richard D. Irwin, Inc., Homewood, Illinois.<br />

McPhers<strong>on</strong>, T.N. and Brown, M.J. (2003) “Estimating Daytime and Nighttime Populati<strong>on</strong> Distributi<strong>on</strong>s In U.S.<br />

Cities for Emergency Resp<strong>on</strong>se Activities”, Preprints: 84 th AMS Annual Meeting, AMS, Seattle, WA, 10 pp.<br />

Mishra, S. (2009) “GIS in Indian retail industry – a strategic tool”, Internati<strong>on</strong>al Journal of Marketing Studies,<br />

Vol. 1, No.1, pp 50-57.<br />

108


Activity Theory: A Useful Evaluati<strong>on</strong> Methodology for the<br />

Role of Informati<strong>on</strong> Systems in Collaborative Activity<br />

Audrey Grace<br />

Business Informati<strong>on</strong> Systems, University College Cork, Ireland<br />

a.grace@ucc.ie<br />

Abstract: The way in which informati<strong>on</strong> systems are used in organisati<strong>on</strong>s has evolved over time. While they<br />

were initially used primarily for informati<strong>on</strong> processing and for supporting company centric efficiencies, they are<br />

now extensively used to share informati<strong>on</strong> and to support collaborati<strong>on</strong> both internally within an organizati<strong>on</strong> and<br />

with external customers, suppliers and partners. While much IS research heretofore has c<strong>on</strong>centrated <strong>on</strong> how<br />

informati<strong>on</strong> systems facilitate informati<strong>on</strong> processing and the decisi<strong>on</strong> making of individuals in an organisati<strong>on</strong>,<br />

there is a growing need within organisati<strong>on</strong>s to analyse and understand how informati<strong>on</strong> systems facilitate both<br />

internal and external informati<strong>on</strong> sharing and collaborati<strong>on</strong>. This paper provides an overview of activity theory and<br />

argues that this theory provides a holistic and insightful evaluati<strong>on</strong> methodology which will allow researchers to<br />

investigate how collaborati<strong>on</strong> is achieved through all elements of IS (people, process, technology). The key<br />

characteristics of activity theory that underpin its suitability for researching collaborati<strong>on</strong> through IS are outlined.<br />

Finally, a specific example of future research using this theory is described.<br />

Keywords: activity theory, evaluati<strong>on</strong> methodology, informati<strong>on</strong> systems, collaborati<strong>on</strong><br />

1. Introducti<strong>on</strong><br />

Historically, informati<strong>on</strong> systems were utilised within an organisati<strong>on</strong>’s boundaries to minimise<br />

operating costs and to improve the efficiency of internal business processes (Alter 1992; Mukherji<br />

2002). The c<strong>on</strong>ceptualisati<strong>on</strong> of the extended enterprise and boundary-less organisati<strong>on</strong> encouraged<br />

managers to broaden their search for efficiencies and discover new ways of creating value from their<br />

supplier network and bey<strong>on</strong>d (Elliott 2001; Prahalad and Ramaswamy 2002).<br />

Indeed, the commercial diffusi<strong>on</strong> of the Internet in the 1990s completely overhauled interorganisati<strong>on</strong>al<br />

integrati<strong>on</strong> as companies began to integrate Internet technologies with their existing<br />

informati<strong>on</strong> systems by c<strong>on</strong>necting a web fr<strong>on</strong>t-end to their internal applicati<strong>on</strong>s (Ash and Burn 2003;<br />

Legner 2008). Externally focused informati<strong>on</strong> systems known as Inter-organisati<strong>on</strong>al Informati<strong>on</strong><br />

Systems (IOS) emerged that transcended organisati<strong>on</strong>al boundaries (H<strong>on</strong>g 2002; Shin 2006) and<br />

enabled informati<strong>on</strong> sharing between organisati<strong>on</strong>s over the Internet (Bakos 1991; Majchrzak et al.<br />

2000). The increasing role of web-based technologies to support all aspects of a company’s business<br />

operati<strong>on</strong>s (i.e. electr<strong>on</strong>ic business or e-business) has been widely acknowledged both by the<br />

research community (Cagliano et al. 2005; cf. Evans and Wurster 1999) and by practiti<strong>on</strong>ers – for<br />

example: e-business reports have been published by many of the large c<strong>on</strong>sulting firms including<br />

Forrester Group (cf. Johns<strong>on</strong> 2003) and Morgan Stanley (cf. Witter 1999).<br />

It is comm<strong>on</strong> to think of e-business as digitally enabled informati<strong>on</strong> sharing to support co-ordinati<strong>on</strong><br />

and collaborati<strong>on</strong> in three main areas: (i) within a business which focuses <strong>on</strong> supporting corporate<br />

activities and the integrati<strong>on</strong> of departmental activities; (ii) between a business and its channel<br />

partners (e.g. suppliers, distributors, or retailers); and (iii) between a business and c<strong>on</strong>sumers, which<br />

includes electr<strong>on</strong>ic shopping, product marketing, informati<strong>on</strong> retrieval, entertainment and client service<br />

activities (Shaw et al. 1997; Wu et al. 2003).<br />

In relati<strong>on</strong> to the co-ordinati<strong>on</strong> within a business, organisati<strong>on</strong>al effectiveness comes from informati<strong>on</strong><br />

sharing to leverage intellect and knowledge in order to achieve corporate objectives rather than<br />

focusing <strong>on</strong> ec<strong>on</strong>omies of scale in operati<strong>on</strong>s or physical sources of advantages (Ash and Burn 2003;<br />

Venkatraman and Henders<strong>on</strong> 1998). The ability of organisati<strong>on</strong>s to make processes effective is<br />

increasingly supported by intranets that facilitate team-level co-ordinati<strong>on</strong> (to achieve team objectives)<br />

and exchange of informati<strong>on</strong> and knowledge (Thomas and Bostrom 2010; Venkatraman and<br />

Henders<strong>on</strong> 1998). With respect to co-ordinati<strong>on</strong> and informati<strong>on</strong> sharing between a business and its<br />

channel partners, business-to-business e-commerce refers to transacti<strong>on</strong>s and informati<strong>on</strong> sharing to<br />

facilitate collaborative processes between organisati<strong>on</strong>s within supply chains (Cullen and Webster<br />

2007; Mahadevan 2003). Regarding the interface between a business and c<strong>on</strong>sumers, because the<br />

web enables organisati<strong>on</strong>s to support a high level of client interacti<strong>on</strong> (Straub and Wats<strong>on</strong> 2001), it<br />

offers a unique opportunity to collaborate with individual clients in order to customise products and<br />

services for them (Straub and Wats<strong>on</strong> 2001; Venkatraman and Henders<strong>on</strong> 1998). This ability offers<br />

109


Audrey Grace<br />

many benefits to an organizati<strong>on</strong>, including increased client satisfacti<strong>on</strong> and client loyalty (Piller and<br />

Muller 2004; Wind and Rangaswamy 2001) as well as an opportunity to protect against<br />

commoditisati<strong>on</strong> through differentiati<strong>on</strong> (Piller and Muller 2004; cf. Wind and Rangaswamy 2001).<br />

Bearing in mind this growing orientati<strong>on</strong> towards the use of informati<strong>on</strong> systems to enable<br />

collaborative activity (both internally and externally), this paper argues that Activity Theory provides a<br />

very useful c<strong>on</strong>ceptual framework for evaluating how effective informati<strong>on</strong> systems are within this<br />

c<strong>on</strong>text. Secti<strong>on</strong> 2 provides an overview of Activity Theory. Secti<strong>on</strong> 3 discusses the growing use of<br />

Activity Theory within a number of fields, including the IS field. Secti<strong>on</strong> 4 discusses the views of<br />

Activity Theory <strong>on</strong> the use of technological and n<strong>on</strong>-technological tools in collaborative activities.<br />

Finally, the appropriateness of activity theory for evaluating the use of informati<strong>on</strong> systems in<br />

collaborative activities, both within and outside the enterprise, is discussed (Secti<strong>on</strong> 5).<br />

2. Overview of activity theory<br />

The theory of activity (Engeström 1987; cf. Le<strong>on</strong>t'ev 1978; Vygotsky 1978) is rooted in culturalhistorical<br />

psychology and may be defined as a “philosophical and cross-disciplinary framework for<br />

studying different forms of human practices as development processes, with both individual and social<br />

levels interlinked at the same time” (Kuutti 1996, p. 25). Activity theory has its origins in the<br />

Vygotskyian (1978) c<strong>on</strong>cept of artefact-mediated and object-oriented acti<strong>on</strong> whereby human beings’<br />

interacti<strong>on</strong>s with their envir<strong>on</strong>ment are not direct, but instead the interacti<strong>on</strong> between a human<br />

individual and the objects of the envir<strong>on</strong>ment is mediated by cultural tools (see Figure 1).<br />

Figure 1: Mediati<strong>on</strong> model (Vygotsky 1978)<br />

Le<strong>on</strong>t’ev (1978) further developed Vygotsky’s ideas of social and cultural mediati<strong>on</strong> by developing a<br />

hierarchical model of human activity. He argued that: (i) a minimal meaningful c<strong>on</strong>text for individual<br />

acti<strong>on</strong>s must be included in the basic unit of analysis (i.e. an activity); and (ii) because the c<strong>on</strong>text is<br />

included in the unit of analysis, the object of research is always essentially collective, even if the main<br />

research interest is in individual acti<strong>on</strong>s (Kuutti 1996).<br />

Inspired by this, Engeström (1987) introduced an expanded versi<strong>on</strong> of the mediati<strong>on</strong> model to reflect<br />

the collective and collaborative nature of human activity. Engeström’s representati<strong>on</strong> of an activity<br />

system (see Figure 2) depicts the relati<strong>on</strong>ship between three key elements of an activity. These are:<br />

(i) the subject of an activity; (ii) other actors involved in the activity (i.e. the community); and (iii) the<br />

shared object of the activity in which they are jointly engaged. These elements are represented by the<br />

inner triangle drawn with broken lines in this figure.<br />

Figure 2 also illustrates three c<strong>on</strong>textual factors that mediate these relati<strong>on</strong>ships – represented by the<br />

extremities of the outer triangle in the figure. They are: (i) roles/resp<strong>on</strong>sibilities (i.e. the divisi<strong>on</strong> of<br />

labour between the subject and all other actors involved in the activity); (ii) rules/norms (i.e. explicit<br />

governing regulati<strong>on</strong>s and implicit social/cultural norms); and (iii) tools (i.e. c<strong>on</strong>cepts, instruments,<br />

language, signs, technologies).<br />

These elements of the model mediate the relati<strong>on</strong>ship between a subject, other actors involved in the<br />

activity (i.e. their work community) and their shared object. Furthermore, the mediating role/rule/tool<br />

artefacts may be created or transformed during the development of the activity and carry with them a<br />

particular culture and historical residue of that development (Engeström and Miettinen 1999a).<br />

110


Audrey Grace<br />

Figure 2: The activity system, adapted from Engeström (1987)<br />

Activity theory holds that the human mind emerges, exists and can <strong>on</strong>ly be understood within the<br />

c<strong>on</strong>text of human interacti<strong>on</strong> with the world; and that this interacti<strong>on</strong> (i.e. the activity) is socially and<br />

culturally determined (Engeström 1999b). Activities are c<strong>on</strong>sidered inherently dynamic because<br />

outcomes are characterised in terms of their dual individual and social existence in the c<strong>on</strong>sciousness<br />

of the performing subject.<br />

Even though some individuals may be more powerful in the collective activity, no <strong>on</strong>e individual can<br />

completely impose his or her view <strong>on</strong> other pers<strong>on</strong>s taking part in the activity. Because of this, it is<br />

useful to view the perspectives of different subjects within the activity, when analysing an activity<br />

(Boer et al. 2002; Virkkunen and Kuutti 2000).<br />

An activity does not exist in a vacuum. Instead it exists as a node in a multi-dimensi<strong>on</strong>al network of<br />

activity systems (Engeström 1992). In the network around a ‘central’ activity, there are typically such<br />

activities as: (i) the object activity/activities (e.g. an object to be further transformed in the value<br />

chain); (ii) supervisory activities (e.g. where rules or divisi<strong>on</strong> of labour for an activity are defined or<br />

rearranged; and (iii) support activities (Engeström 1992; Kuutti and Molin-Juustila 1998), for example:<br />

where tools and processes for an activity are developed or refined. The activity theory approach<br />

emphasises the incoherencies, inc<strong>on</strong>sistencies and tensi<strong>on</strong>s that exist within an activity system itself<br />

and between an activity system and a neighbouring activity in its network (cf. Engeström 1987;<br />

Wiredu and Sorensen 2006). As a result, an activity system is c<strong>on</strong>stantly developing and working<br />

through tensi<strong>on</strong>s within and between its comp<strong>on</strong>ents and is also evolving collectively with other<br />

activity systems in the network.<br />

3. Growing popularity of activity theory<br />

Activity theory which was first introduced to the IS community by Bodker (1989; 1991) has the<br />

potential to inform a much broader perspective <strong>on</strong> real-life uses of technology than the traditi<strong>on</strong>al<br />

cognitive approach (cf. Kaptelinin and Nardi 2006; Korpela et al. 2004; Kuutti 1991). Activity theory<br />

provides a very useful analytical framework for understanding and analysing the mediati<strong>on</strong> of human<br />

activities by both technological and n<strong>on</strong>-technological artefacts (Bardram 1998a).<br />

In recent years, activity theory has been applied extensively to research in the field of informati<strong>on</strong><br />

systems (cf. Anth<strong>on</strong>y 2012; Hasan and Gould 2001; Huang 2011). Activity theory is also becoming<br />

increasingly popular in a number of other fields. For example it has been extensively used in the field<br />

of human-computer interacti<strong>on</strong> (cf. Bann<strong>on</strong> and Bodker 1991; Mwanza 2002) and computersupported<br />

cooperative work (cf. Collins et al. 2002; Korpela and Soriyan 1998).<br />

Researchers have also widely drawn up<strong>on</strong> this theory in studies <strong>on</strong> organisati<strong>on</strong>al learning (cf.<br />

Ah<strong>on</strong>en and Virkkunen 2003; Engeström 2004) and electr<strong>on</strong>ic learning (cf. Kaptelinin and Cole 2002).<br />

111


Audrey Grace<br />

Several internati<strong>on</strong>al journals have published special issues devoted to studies based <strong>on</strong> activity<br />

theory, including the Scandinavian Journal of Informati<strong>on</strong> Systems in 2000, Computer Supported<br />

Cooperative Work in 1999 and 2002 and Interacting with Computers in 2003.<br />

4. Activity theory <strong>on</strong> the use of tools<br />

According to Activity Theory, a tool or artifact provides a means or instrument for carrying out an<br />

activity (Mwanza 2002; Wertsch 1997). Tools embody cultural knowledge (Kuutti 1996) as well as the<br />

formulisati<strong>on</strong> of work practices (Christiansen 1996). They are collectively generated and maintained<br />

(Kaptelinin and Nardi 2006). Furthermore, the c<strong>on</strong>text of an activity and the individual habits of the<br />

subject may encourage the use of <strong>on</strong>e tool over another (Wertsch 1997).<br />

Tools can expand our possibilities to manipulate and transform different objects, but <strong>on</strong> the other<br />

hand, the object may be perceived and manipulated within the limitati<strong>on</strong>s set by the tool (Bann<strong>on</strong><br />

1997). Thus, a particular tool may be both enabling and limiting in that it provides a set of opti<strong>on</strong>s from<br />

established patterns of achieving the object of the activity, yet it restricts the interacti<strong>on</strong> to be from the<br />

perspective of that particular tool (Kuutti 1996; Wertsch 1997). Put simply, a tool works well in our<br />

activity if it allows us to focus our attenti<strong>on</strong> <strong>on</strong> the real object and badly if it doesn’t (Bann<strong>on</strong> and<br />

Bodker 1991).<br />

Furthermore, Bann<strong>on</strong> and Bodker emphasise that tools should <strong>on</strong>ly be c<strong>on</strong>sidered from the<br />

perspective of their c<strong>on</strong>text (i.e. their actual use). Referring to the notebooks of Samuel Butler (1835-<br />

1902), they cite his example of a complex machine, if intended for use by children, ceases to be a tool<br />

and becomes a toy. They add that “it is seriousness of aim and recogniti<strong>on</strong> of suitability for the<br />

achievement of that aim and not anything in the tool itself that makes the tool” (Bann<strong>on</strong> and Bodker<br />

1991, p. 227).<br />

Following the work of Vygotsky (1978), Engeström (1987) distinguishes between technical tools which<br />

are directed toward the c<strong>on</strong>trol of processes of nature and psychological tools/signs which are<br />

directed toward the mastery or c<strong>on</strong>trol of behavioural processes. Examples of psychological tools and<br />

their complex systems include: language, various systems for counting, schemes and c<strong>on</strong>venti<strong>on</strong>al<br />

signs (Vygotsky 1978). Technical tools include physical instruments or artefacts and may be either<br />

technological or n<strong>on</strong>-technological.<br />

Following the work of Latour (1993), Kaptelinin (1996b) emphasises that the role of technical tools is<br />

not limited to transmissi<strong>on</strong> of operati<strong>on</strong>al aspects of human interacti<strong>on</strong> with the world. He argues that<br />

implicit goals built into technical tools can also shape the goals of the people who use these tools.<br />

Technical tools serve the double purpose of both doing something for you and reminding you of<br />

something you can do (Christiansen 1996).<br />

Activity theory holds that tacit knowledge is gradually formalised into culturally created technological<br />

artefacts and that these artefacts mediate subsequent iterati<strong>on</strong>s of the activity (Kaptelinin et al. 1999).<br />

Engeström (1999b) also puts forward the noti<strong>on</strong> that technical tools or mediating artefacts are integral<br />

and inseparable comp<strong>on</strong>ents of human functi<strong>on</strong>ing. The idea is that humans can c<strong>on</strong>trol their own<br />

behaviour – not ‘from the inside’, <strong>on</strong> the basis of biological urges, but ‘from the outside’, using and<br />

creating artefacts (Engeström 1999b). He asserts that this perspective presents an opportunity for the<br />

“serious study of artefacts as integral and inseparable comp<strong>on</strong>ents of human functi<strong>on</strong>ing” (Engeström<br />

1999b, p. 29).<br />

5. Discussi<strong>on</strong> and c<strong>on</strong>clusi<strong>on</strong>s<br />

Activity theory provides an insightful evaluati<strong>on</strong> methodology that will help to improve our scholarly<br />

understanding of how informati<strong>on</strong> systems enable informati<strong>on</strong> sharing and collaborati<strong>on</strong> both within<br />

the enterprise and across enterprise boundaries for a number of reas<strong>on</strong>s (see Table 1).<br />

First, the ‘people/process/technology’ nature of the Informati<strong>on</strong> Systems (cf. Duff and Assad 1980;<br />

Keen 1993) is captured in activity theory (see Secti<strong>on</strong> 2). In additi<strong>on</strong>, activity theory also incorporates<br />

a pragmatic focus <strong>on</strong> how the objective of the collaborative activity is achieved through these three<br />

elements of IS. Activity theory, thus, provides an excellent c<strong>on</strong>ceptual vehicle for evaluating how<br />

technological informati<strong>on</strong> systems enable the objectives of collaborative activities to be reached while<br />

also remaining cognisent of the collaborative roles that people play in the activity and the rules/norms<br />

that guide these individuals as they interact together to achieve particular goals.<br />

112


Audrey Grace<br />

Table 1: Usefulness of activity theory for analysing how IS supports collaborative activity<br />

Characteristic of Activity<br />

Theory (AT)<br />

AT provides a c<strong>on</strong>ceptual<br />

framework that allows<br />

researchers to analyse how<br />

three mediating factors (rules,<br />

roles & tools) mediate the<br />

relati<strong>on</strong>ship between a subject,<br />

other actors involved in a<br />

collaborative activity and their<br />

shared objective (Engeström<br />

1987)<br />

AT focuses <strong>on</strong> informati<strong>on</strong><br />

sharing in a system rather than<br />

individual informati<strong>on</strong><br />

processing (Korpela et al. 2004)<br />

AT incorporates both<br />

technological and n<strong>on</strong>technological<br />

tools (Kuutti 1991)<br />

AT emphasises the importance<br />

of studying the c<strong>on</strong>text within<br />

which informati<strong>on</strong> systems are<br />

used, including the social<br />

c<strong>on</strong>text (Crawford and Hasan<br />

2006; Kaptelinin and Nardi<br />

2006)<br />

Relevance to how IS facilitates<br />

Collaborative Activity<br />

Incorporates the three key<br />

elements of IS (i.e. people, process<br />

and technology) with a str<strong>on</strong>g<br />

emphasis <strong>on</strong> how the objective of<br />

the collaborative activity is<br />

achieved<br />

Collaborative activities, by their<br />

nature, are underpinned by<br />

informati<strong>on</strong> sharing (Jagdev and<br />

Thoben 2001; Sarker et al. 2000)<br />

There is clear evidence that<br />

informati<strong>on</strong> systems are not being<br />

used to their full potential in<br />

collaborative activity (Mawer et al.<br />

2010; Muñoz-Ericks<strong>on</strong> et al. 2010)<br />

Collaborative activities are<br />

inherently socio-technical systems<br />

(de Moor and Weigand 2007; Ritter<br />

et al. 2007)<br />

Potential Future Research<br />

Use the collaborative activity as<br />

the basic unit of analysis in order<br />

to analyse how the objective of a<br />

collabortive activity is enabled<br />

through people, process and<br />

technology<br />

Use AT to specifically investigate<br />

how informati<strong>on</strong> systems facilitate<br />

informati<strong>on</strong> sharing between<br />

people involved in a particular<br />

collaborative activity<br />

Use AT to identify areas where<br />

informati<strong>on</strong> systems could play a<br />

greater role in facilitating<br />

collaborati<strong>on</strong><br />

Use AT to study how informati<strong>on</strong><br />

systems compliment/c<strong>on</strong>flict with<br />

the roles that the people play or<br />

the rules that guide these people<br />

in the collaborative activity<br />

Sec<strong>on</strong>d, at the heart of activity theory is emphasis focuses <strong>on</strong> informati<strong>on</strong> sharing between people in<br />

a system, rather than focusing <strong>on</strong> individual informati<strong>on</strong> processing, as is the case in many IS studies<br />

(cf. Korpela et al. 2004; Kuutti 1991). The mediated approach adopted by activity theory is, therefore,<br />

useful in analysing how people share informati<strong>on</strong> with each other as they collaborate together. This is<br />

particularly pertenent as the employment of informati<strong>on</strong> systems shifts from informati<strong>on</strong> processing to<br />

informati<strong>on</strong> sharing and collaborati<strong>on</strong>. Third, there is clear evidence that informati<strong>on</strong> systems are not<br />

being used to their full potential in collaborative activity (Mawer et al. 2010; Muñoz-Ericks<strong>on</strong> et al.<br />

2010). Because activity theory incorporates the study of both technological and n<strong>on</strong>-technological<br />

tools in enabling collaborative activities (Kuutti 1991), this theory could be used by researchers to<br />

identify the limitati<strong>on</strong>s of informati<strong>on</strong> systems in supporting collaborative activities, or to uncover areas<br />

where informati<strong>on</strong> systems could augment or replace n<strong>on</strong>-technological tools in such circumstances.<br />

Fourth, activity theory emphasises the importance of studying the c<strong>on</strong>text within which technology is<br />

used, including the social c<strong>on</strong>text (Crawford and Hasan 2006; Kaptelinin and Nardi 2006). This makes<br />

activity theory particularly useful for analysing how informati<strong>on</strong> systems are used to share informati<strong>on</strong><br />

between parties who are attempting to collaborate in a social setting. Crawford and Hasan (2006, p.<br />

54) argue that “complex phenomena associated with socio-technical systems...are prime targets for<br />

research using activity theory because activity theory provides a framework for emerging patterns of<br />

human activity in terms of changing purposes, awareness, focus of attenti<strong>on</strong> and tools”. In c<strong>on</strong>clusi<strong>on</strong>,<br />

this paper has underlined the need for an improved scholarly understanding of how informati<strong>on</strong><br />

systems support informati<strong>on</strong> sharing and collaborati<strong>on</strong> within and across organizati<strong>on</strong>al boundaries. It<br />

has illustrated the suitability of activity theory as an evaluati<strong>on</strong> methodology for further research in this<br />

area. Based <strong>on</strong> a number of key characteristics of activity theory, the paper highlights a number of<br />

areas which would benefit from the applicati<strong>on</strong> of this theory. Activity theory provides a holistic and<br />

insightful c<strong>on</strong>ceptual vehicle which will allow researchers to investigate and evaluate how<br />

collaborati<strong>on</strong> is achieved through all elements of IS (people, process, technology). It can also be<br />

employed to specifically examine how informati<strong>on</strong> systems support informati<strong>on</strong> sharing between<br />

people who are collaborating together. Used as an evaluati<strong>on</strong> methodology, activity theory can be<br />

used by researchers to unearth the limitati<strong>on</strong>s of informati<strong>on</strong> systems in supporting collaborative<br />

activities and to identify areas where informati<strong>on</strong> systems could play an increased role in such<br />

c<strong>on</strong>texts.<br />

113


Audrey Grace<br />

For example, the researcher is about to undertake a qualitative research project to study how<br />

collaborative decisi<strong>on</strong>s are made between General Practiti<strong>on</strong>ers (GPs) and patients with multiple<br />

chr<strong>on</strong>ic diseases <strong>on</strong> appropriate healthcare for the patient (see Figure 3). Because these decisi<strong>on</strong>s<br />

may involve trade-offs between guidelines/treatments for various diseases, patient preferences are<br />

relevant and decisi<strong>on</strong>s are thus collaboratively made between GPs and patients. The aim of the<br />

project is to use activity theory to identify, operati<strong>on</strong>alise and evaluate how informati<strong>on</strong> systems could<br />

play a greater role in facilitating collaborative decisi<strong>on</strong>s in this specific c<strong>on</strong>text.<br />

Figure 3: The doctor/patient activity system, adapted from Engeström (1987)<br />

It is planned to first review the current activity system whereby GPs and patients collaborate to decide<br />

<strong>on</strong> appropriate medicati<strong>on</strong>; elective procedures; necessary lifestyle adjustments; etc. It is then<br />

proposed to design and develop a new pilot decisi<strong>on</strong> support system to aid informati<strong>on</strong> sharing and<br />

collaborati<strong>on</strong> between the GP and the patient. The design of this system will be informed by reviewing<br />

how the collaborative decisi<strong>on</strong>s are currently mediated by tools (both technological and n<strong>on</strong>technological),<br />

the roles that GPs and patients play in making the collaborative decisi<strong>on</strong>s, as well as<br />

the rules that guide such collaborative decisi<strong>on</strong>s. Existing decisi<strong>on</strong> aids and paper-based<br />

rules/guidelines for individual chr<strong>on</strong>ic diseases will be coded into the new system. Any tradeoffs/c<strong>on</strong>flicts<br />

that exist between guidelines for different c<strong>on</strong>diti<strong>on</strong>s will be highlighted by the new<br />

system.<br />

A number of GPs will then be asked to use the pilot system and the transformed activity system (i.e.<br />

incorporating the new decisi<strong>on</strong> support system) will be analysed. Activity theory will be used to<br />

evaluate how the objective of the activity (the collaborative decisi<strong>on</strong> between the GP and the patient)<br />

is currently achieved through people, process and technology, versus how it is achieved with the<br />

introducti<strong>on</strong> of the new decisi<strong>on</strong> support system. Drawing <strong>on</strong> activity theory, the researcher will<br />

investigate if the actors involved in the activity (the GP and the patient) acquire an improved<br />

understanding of the relevant clinical rules/guidelines that should guide collaborative decisi<strong>on</strong> in the<br />

case of multimorbidity. The researcher will also analyse if the introducti<strong>on</strong> of the new system alters the<br />

roles that either party plays or alters the speed/nature of the collaborative decisi<strong>on</strong> that is reached.<br />

References<br />

Ah<strong>on</strong>en, H., and Virkkunen, J. "Shared Challenge for Learning: Dialogue between <strong>Management</strong> and Fr<strong>on</strong>t-line<br />

Workers in Knowledge <strong>Management</strong>," Informati<strong>on</strong> Technology and <strong>Management</strong> (2:1/2) 2003, pp 59-84.<br />

Alter, S. Informati<strong>on</strong> Systems: A <strong>Management</strong> Perspective Addis<strong>on</strong>-Wesley, Menlo Park, California, 1992.<br />

Anth<strong>on</strong>y, A. "Activity Theory as a Framework for Investigating District-Classroom System Interacti<strong>on</strong>s and Their<br />

Influences <strong>on</strong> Technology Integrati<strong>on</strong>," Journal of Research <strong>on</strong> Technology in Educati<strong>on</strong> (44:4),<br />

Summer2012 2012, pp 335-356.<br />

Ash, C., and Burn, J. "A Strategic Framework for the <strong>Management</strong> of ERP enabled E-business Change,"<br />

<str<strong>on</strong>g>European</str<strong>on</strong>g> Journal of Operati<strong>on</strong>s Research (146:2) 2003, pp 374-387.<br />

Bakos, J. "A Strategic Analysis of Electr<strong>on</strong>ic Marketplaces," MIS Quarterly (15:3) 1991, pp 295-310.<br />

114


Audrey Grace<br />

Bann<strong>on</strong>, L. "Activity Theory," 1997.<br />

Bann<strong>on</strong>, L., and Bodker, S. "Bey<strong>on</strong>d the Interface: Encountering Artifacts in Use," in: Designing Interacti<strong>on</strong>:<br />

Psychology at the Human-Computer Interface, J. Carroll (ed.), Cambridge University Press, New York,<br />

1991, pp. 227-253.<br />

Bardram, J. "Designing for the Dynamics of Co-operative Work Activities," Proceedings of the 1998 ACM<br />

<str<strong>on</strong>g>C<strong>on</strong>ference</str<strong>on</strong>g> <strong>on</strong> Computer-Supported Cooperative Work, ACM Press, Seattle, 1998a, pp. 89-98.<br />

Bodker, S. "A Human Activity Approach to User Interfaces," Human Computer Interacti<strong>on</strong> (4) 1989, pp 171-195.<br />

Bodker, S. Through the Interface: A Human Activity Approach to User Interface Design Lawrence Erlbaum,<br />

Hillsdale, NJ., 1991.<br />

Boer, N., Baalen, P., and Kuman, K. "An Activity Theory Approach for Studying the Situatedness of Knowledge<br />

Sharing," Proceedings of the 35th Internati<strong>on</strong>al <str<strong>on</strong>g>C<strong>on</strong>ference</str<strong>on</strong>g> <strong>on</strong> System Sciences, Hawaii, 2002.<br />

Butler, S. "The Note-Books of Samuel Butler," in: EBook 6173, D. Price (ed.), Project Gutenberg, , 1835-1902.<br />

Cagliano, R., Caniato, F., and Spina, G. "E-business Strategy: How Companies are shaping their Supply Chain<br />

through the Internet," Internati<strong>on</strong>al Journal of Operati<strong>on</strong>s and Producti<strong>on</strong> <strong>Management</strong> (25:12) 2005, pp<br />

1309-1327.<br />

Christiansen, E. "Tamed By a Rose: Computers as Tools in Human Activity," in: C<strong>on</strong>text and C<strong>on</strong>sciousness:<br />

Activity Theory and Human-Computer Interacti<strong>on</strong>, B. Nardi (ed.), The MIT Press, Cambridge,<br />

Massachusetts, 1996, pp. 175-198.<br />

Collins, P., Shulka, S., and Redmiles, D. "Activity Theory and System Design: A View from the Trenches,"<br />

Computer Supported Cooperative Work (11:1/2) 2002, pp 55-80.<br />

Crawford, K., and Hasan, H. "Dem<strong>on</strong>strati<strong>on</strong>s of the Activity Theory Framework for Research in Informati<strong>on</strong><br />

Systems," Australasian Journal of Informati<strong>on</strong> Systems (13:2), May 2006, pp 49-68.<br />

Cullen, A., and Webster, M. "A model of B2B e-commerce based <strong>on</strong> c<strong>on</strong>nectivity and purpose," Internati<strong>on</strong>al<br />

Journal of Operati<strong>on</strong>s and Producti<strong>on</strong> <strong>Management</strong> (27:2) 2007, pp 205-225.<br />

de Moor, A., and Weigand, H. "Formalizing the evoluti<strong>on</strong> of virtual communities," Informati<strong>on</strong> Systems (32:2)<br />

2007, pp 223-247.<br />

Duff, W., and Assad, M. Informati<strong>on</strong> <strong>Management</strong>: An Executive Approach Oxford University Press, L<strong>on</strong>d<strong>on</strong>,<br />

1980.<br />

Elliott, S. "Collaborative Advantage: Winning Through Extended Enterprise Supplier Networks (Book)," Journal of<br />

Product Innovati<strong>on</strong> <strong>Management</strong> (18:5) 2001, pp 352-352.<br />

Engeström, Y. Learning by Expanding: An activity-Theoretical Approach to Developmental Research Orienta-<br />

K<strong>on</strong>sultit, Helsinki, 1987.<br />

Engeström, Y. "Interactive Expertise: Studies in Distributed Working Intelligence. Research Bulletin 83,"<br />

Department of Educati<strong>on</strong>, University of Helsinki, pp. 3-83.<br />

Engeström, Y. "Activity and Individual and Social Transformati<strong>on</strong>," in: Perspectives <strong>on</strong> Activity Theory, Y.<br />

Engeström, R. Miettinen and R. Punamaki (eds.), Cambridge University Press, Cambridge, 1999b, pp. 19-<br />

38.<br />

Engeström, Y. "New Forms of Learning in Co-C<strong>on</strong>figurati<strong>on</strong> Work," Journal of Workplace Learning (16:1/2) 2004,<br />

pp 11-21.<br />

Engeström, Y., and Miettinen, R. "Introducti<strong>on</strong>," in: Perspectives <strong>on</strong> Activity Theory, Y. Engeström, R. Miettinen<br />

and R. Punamaki (eds.), Cambridge University Press, Cambridge, 1999a, pp. 1-16.<br />

Evans, P., and Wurster, T. "Getting real about Virtual Commerce," Harvard Business Review (77:6) 1999, pp 84-<br />

94.<br />

Hasan, H., and Gould, E. "Support for the sense-making activity of managers," Decisi<strong>on</strong> Support Systems (31)<br />

2001, pp 71-86.<br />

H<strong>on</strong>g, I. "A new Framework for Interorganisati<strong>on</strong>al systems based <strong>on</strong> the linkage of participants' roles,"<br />

Informati<strong>on</strong> & <strong>Management</strong> (4:2) 2002, pp 261-270.<br />

Huang, K.-H. "Learning in Authentic C<strong>on</strong>texts: Projects Integrating Spatial Technologies and Fieldwork," Journal<br />

of Geography in Higher Educati<strong>on</strong> (35:4) 2011, pp 565-578.<br />

Jagdev, H. S., and Thoben, K. D. "Anatomy of enterprise collaborati<strong>on</strong>s," Producti<strong>on</strong> Planning & C<strong>on</strong>trol (12:5)<br />

2001, pp 437-451.<br />

Johns<strong>on</strong>, C. "Highlight: US e-Commerce to Hit Nearly $230 Billi<strong>on</strong> in 2008," p.<br />

http://www.forrester.com/rb/Research/highlight_us_ecommerce_hits_$230_billi<strong>on</strong>_in_2008/q/id/17217/t/172<br />

12.<br />

Kaptelinin, V. "Computer-Mediated Activity: Functi<strong>on</strong>al Organs in Social and Developmental C<strong>on</strong>texts," in:<br />

C<strong>on</strong>text and C<strong>on</strong>sciousness: Activity Theory and Human Computer Interacti<strong>on</strong>, B. Nardi (ed.), The MIT<br />

Press, Cambridge, Massachusetts, 1996b, pp. 45-68.<br />

Kaptelinin, V., and Cole, M. "Individual and Collective Activities in Educati<strong>on</strong>al Computer Game Playing," in:<br />

CSCL 2: Carrying Forward the C<strong>on</strong>versati<strong>on</strong>, T. Koschmann, R. Hall and N. Miyake (eds.), Lawrence<br />

Erlbaum, Mahway, NJ, 2002.<br />

Kaptelinin, V., and Nardi, B. Acting With Technology: Activity Theory and Interacti<strong>on</strong> Design The MIT Press,<br />

Cambridge, Massachusetts, 2006.<br />

Kaptelinin, V., Nardi, B., and Macaulay, C. "Methods and Tools: The activity Checklist: A Tool for Representing<br />

the 'Space' of C<strong>on</strong>text," Interacti<strong>on</strong>s), July-August 1999, pp 27-39.<br />

Keen, P. G. "Informati<strong>on</strong> Technology and the <strong>Management</strong> Difference: A Fusi<strong>on</strong> Map," IBM Systems Journal<br />

(32:1) 1993, pp 17-39.<br />

115


Audrey Grace<br />

Korpela, M., Mursu, A., Soriyan, H., Eerola, A., Hakkinen, H., and Toivanen, M. "Informati<strong>on</strong> Systems Research<br />

and Development By Activity Analysis and Development: Dead Horse or the Next Wave?," in: Informati<strong>on</strong><br />

Systems Research: Relevant Theory and Informed Practice: IFIP TC8 WG8.2, B. Kaplan (ed.), Kluwer,<br />

L<strong>on</strong>d<strong>on</strong> 2004, pp. 453-471.<br />

Korpela, M., and Soriyan, H. "Community participati<strong>on</strong> in Health Informatics in Africa: An Experiment in Tripartite<br />

Partnership in Lle-Lfe, Nigeria," Computer Supported Cooperative Work (7:3-4) 1998, pp 341-361.<br />

Kuutti, K. "Activity Theory and its Applicati<strong>on</strong>s to Informati<strong>on</strong> Systems Research and Development," in:<br />

Informati<strong>on</strong> Systems Research Arena of the 90's, H. Nissen, H. Klein and R. Hirschheim (eds.), North<br />

Holland, Amsterdam, 1991, pp. 525-549.<br />

Kuutti, K. "Activity Theory as a potential framework for human-computer interacti<strong>on</strong> research," in: C<strong>on</strong>text and<br />

C<strong>on</strong>sciousness: Activity Theory and Human Computer Interacti<strong>on</strong>, B. Nardi (ed.), The MIT Press,<br />

Cambridge, Massachusetts, 1996, pp. 17-44.<br />

Kuutti, K., and Molin-Juustila, T. "Informati<strong>on</strong> System Support for Loose Co-ordinati<strong>on</strong> in a Network Organizati<strong>on</strong>:<br />

An Activity Theory Perspective," in: Informati<strong>on</strong> Systems and Activity Theory: Tools in C<strong>on</strong>text, H. Hasan, E.<br />

Gould and P. Hyland (eds.), University of Woll<strong>on</strong>g<strong>on</strong>g Press, Woll<strong>on</strong>g<strong>on</strong>g, Australia, 1998, pp. 73-92.<br />

Latour, B. "On Technical Mediati<strong>on</strong>: The Messenger Lectures <strong>on</strong> the Evoluti<strong>on</strong> of Civilizati<strong>on</strong>, Cornell University,<br />

April 1993," Lund University., Lund, Sweden.<br />

Legner, C. "The Evoluti<strong>on</strong> of B2B E-Services from First Generati<strong>on</strong> E-Commerce Soluti<strong>on</strong>s to Multichannel<br />

Architectures," Journal of Electr<strong>on</strong>ic Commerce in Organizati<strong>on</strong>s (6:2) 2008, pp 58-76.<br />

Le<strong>on</strong>t'ev, A. Activity, C<strong>on</strong>ciousness and Pers<strong>on</strong>ality Prentice-Hall, Englewood Cliffs, NJ, 1978.<br />

Mahadevan, B. "Making Sense of Emerging Market Structures in B2B e-commerce," California <strong>Management</strong><br />

Review (46:1) 2003, pp 86-100.<br />

Majchrzak, A., Rice, R. E., Malhotra, A., King, N., and Ba, S. "Technology Adaptati<strong>on</strong>: The Case of a Computer-<br />

Supported Inter-Organizati<strong>on</strong>al Virtual Team," MIS Quarterly (24:4) 2000, pp 569-600.<br />

Mawer, A. J., Ng, W., and Jacks<strong>on</strong>, M. D. "Collaborati<strong>on</strong> and Informati<strong>on</strong> Sharing Results in Improved Failure<br />

Analysis Tools, Techniques, and Outcomes," Electr<strong>on</strong>ic Device Failure Analysis (12:4) 2010, pp 44-46.<br />

Mukherji, A. "The Evoluti<strong>on</strong> of Informati<strong>on</strong> Systems: Their impact <strong>on</strong> Organizati<strong>on</strong>s and Structures," <strong>Management</strong><br />

Decisi<strong>on</strong> (40:5) 2002, pp 497-506.<br />

Muñoz-Ericks<strong>on</strong>, T. A., Cutts, B. B., Lars<strong>on</strong>, E. K., Darby, K. J., Neff, M., Bolin, B., and Wutich, A. "Spanning<br />

Boundaries in an Ariz<strong>on</strong>a Watershed Partnership: Informati<strong>on</strong> Networks as Tools for Entrenchment or Ties<br />

for Collaborati<strong>on</strong>?," Ecology & Society (15:3) 2010, pp 1-22.<br />

Mwanza, D. "Towards an Activity-Oriented Design Method For HCI Research and Practice," Open University,<br />

Milt<strong>on</strong> Keans, 2002.<br />

Piller, F., and Muller, M. "A New Marketing Approach to Mass Customisati<strong>on</strong>," Internati<strong>on</strong>al Journal of Computer<br />

Integrated Manufacturing (17:7) 2004, pp 583-593.<br />

Prahalad, C., and Ramaswamy, V. "The Co-Creati<strong>on</strong> C<strong>on</strong>necti<strong>on</strong>," Strategy and Business (27) 2002, pp 1-12.<br />

Ritter, J., Ly<strong>on</strong>s, J. B., and Swindler, S. D. "Large-scale coordinati<strong>on</strong>: developing a framework to evaluate sociotechnical<br />

and collaborative issues," Cogniti<strong>on</strong>, Technology & Work (9:1) 2007, pp 33-38.<br />

Sarker, S., Lau, F., and Sahay, S. "Building an inductive theory of collaborati<strong>on</strong> in virtual teams: An adapted<br />

grounded theory approach," in: Proceedings of the 33rd Hawaii Internati<strong>on</strong>al <str<strong>on</strong>g>C<strong>on</strong>ference</str<strong>on</strong>g> <strong>on</strong> System<br />

Sciences, 2000.<br />

Shaw, M., Gardner, D., and Thomas, H. "Research Opportunities in electr<strong>on</strong>ic commerce," Decisi<strong>on</strong> Support<br />

Systems (21) 1997, pp 149-156.<br />

Shin, D. "Distributed Inter-Organizati<strong>on</strong>al Systems and Innovati<strong>on</strong> Processes," Internet Research (16:5) 2006, pp<br />

553-572.<br />

Straub, D., and Wats<strong>on</strong>, H. J. "Research Commentary: Transformati<strong>on</strong>al Issues in Researching IS and Net-<br />

Enabled Organizati<strong>on</strong>s," Informati<strong>on</strong> Systems Research (12:4) 2001, pp 337-345.<br />

Thomas, D. M., and Bostrom, R. P. "Vital Signs for Virtual Teams: An Empirically Developed Trigger Model for<br />

Technology Adapti<strong>on</strong> Interventi<strong>on</strong>s," MIS Quarterly (34:1) 2010, pp 115-142.<br />

Venkatraman, N., and Henders<strong>on</strong>, J. "Real Strategies for Virtual Organising," Sloan <strong>Management</strong> Review (40:1),<br />

Fall 1998, pp 33-48.<br />

Virkkunen, J., and Kuutti, K. "Understanding Organizati<strong>on</strong>al Learning by Focusing <strong>on</strong> Activity Systems,"<br />

Accounting, <strong>Management</strong> and Informati<strong>on</strong> Technologies (10) 2000, pp 291-319.<br />

Vygotsky, L. Mind in Society: The Development of Higher Psychological Processes Harvard University Press,<br />

Cambridge, 1978.<br />

Wertsch, J. "Collective Memory: Issues from a Sociohistorical Perspective," in: Mind, Culutre, and Activity:<br />

Seminal Papers from the Laboratory of Comparative Human Cogniti<strong>on</strong>, M. Cole, R. Engeström and O.<br />

Vasquez (eds.), Cambridge University Press, Cambridge, UK, 1997, pp. 226-232.<br />

Wind, J., and Rangaswamy, A. "Customerizati<strong>on</strong>: The Next Revoluti<strong>on</strong> in Mass Customizati<strong>on</strong>," Journal of<br />

Interactive Marketing (15:1) 2001, pp 13-32.<br />

Wiredu, G., and Sorensen, C. "The Dynamics of C<strong>on</strong>trol and Mobile Computing in Distributed Activities,"<br />

<str<strong>on</strong>g>European</str<strong>on</strong>g> Journal of Informati<strong>on</strong> Systems (15:3) 2006, pp 307-319.<br />

Witter, D. "The <str<strong>on</strong>g>European</str<strong>on</strong>g> Internet Report, Industry Report by Morgan Stanley," 1999.<br />

Wu, F., Mahajan, V., and Balasubramanian, S. "An Analysis of E-Business Adopti<strong>on</strong> and its Impact <strong>on</strong> Business<br />

Performance," Academy of Marketing Science Journal (31:4) 2003, pp 425-447.<br />

116


Dealing With Uncertainty Through KM: Cases in Four<br />

Software SMEs<br />

Ciara Heavin and Frederic Adam<br />

Business Informati<strong>on</strong> Systems, University College Cork, Ireland<br />

c.heavin@ucc.ie<br />

fadam@afis.ucc.ie<br />

Abstract: In the current climate, preparing for change is an issue for companies large and small. For a Small to<br />

Medium Sized Enterprises (SMEs) where resources are significantly limited, it is imperative that efficient<br />

practices are in place to leverage the wealth of knowledge available both inside and outside the firm. It is vital<br />

that these organisati<strong>on</strong>s are swift and flexible enough to survive in this dynamic envir<strong>on</strong>ment, this includes<br />

developing the ability to take stock of the sources and types of knowledge that are valuable to them and<br />

understanding how it is accessed and integrated into the firm’s body of knowledge. C<strong>on</strong>sidering the ec<strong>on</strong>omic<br />

turbulence, never has it been more important to focus <strong>on</strong> the knowledge capabilities of software SMEs, as it is <strong>on</strong><br />

the back of these types of small high-tech organisati<strong>on</strong>s that innovati<strong>on</strong>, growth and potential recovery will be<br />

achieved. Using a qualitative analysis approach in four Irish software SMEs, this study identifies sources of<br />

knowledge and occurrences of knowledge activities (KAs) as a means of understanding the firm’s approach to<br />

knowledge management (KM) and how this may be leveraged therefore providing them with the flexibility to deal<br />

with envir<strong>on</strong>mental uncertainty.<br />

Keywords: knowledge, knowledge management (KM), small and medium sized enterprises (SMEs), knowledge<br />

activity (KA) and software<br />

1. Introducti<strong>on</strong><br />

Defining data, informati<strong>on</strong> and knowledge as distinct and independent phenomena is an demanding<br />

endeavour. In particular it is noted that many authors use the terms informati<strong>on</strong> and knowledge<br />

interchangeably, those (Dennis, Earl, El Sawy, Huber) that c<strong>on</strong>sidered organisati<strong>on</strong>al informati<strong>on</strong><br />

processing in the 1970s, 1980s and early 1990s now focus their attenti<strong>on</strong>s <strong>on</strong> KM as an<br />

organisati<strong>on</strong>al strategy. Figure 1 represents data, informati<strong>on</strong> and knowledge as a c<strong>on</strong>tinuum.<br />

Figure 1: Knowledge c<strong>on</strong>tinuum (after Davenport and Prusak, 1998; Wurman, 2001)<br />

In Figure 1, it is evident that the extremes of each phenomen<strong>on</strong> are distinct however there is<br />

significant overlap between data/informati<strong>on</strong> and informati<strong>on</strong>/knowledge. According to Davenport and<br />

Prusak (1998, p147) “the distincti<strong>on</strong> between knowledge and informati<strong>on</strong> is seen as more of a<br />

c<strong>on</strong>tinuum than a sharp dichotomy. Most projects that focus <strong>on</strong> internal knowledge [repository] deal<br />

with the middle of the c<strong>on</strong>tinuum-informati<strong>on</strong> that represents knowledge to certain users”. Alavi and<br />

Leidner (2001, p109) posit that “informati<strong>on</strong> is c<strong>on</strong>verted to knowledge <strong>on</strong>ce it is processed in the<br />

minds of individuals” while “knowledge becomes informati<strong>on</strong> <strong>on</strong>ce it is articulated and presented in the<br />

form of text, graphics, words or other symbolic forms”. The point where informati<strong>on</strong> becomes<br />

knowledge and vice versa is difficult to pinpoint with complete accuracy, however there is no doubt<br />

that these phenomena are closely linked. In order to adequately observe and measure knowledge in<br />

an organisati<strong>on</strong>, it is essential that an operati<strong>on</strong>al definiti<strong>on</strong> is established. Supporting the view of<br />

Davenport and Prusak (1998) and the point indicated by the arrow in Figure 1, for this study<br />

knowledge occurs when –<br />

117


Ciara Heavin and Frederic Adam<br />

Informati<strong>on</strong> represents valuable knowledge to a group focused <strong>on</strong> achieving a particular task<br />

This definiti<strong>on</strong> is used to identify instances or occurrences of individual knowledge types. From a<br />

practical perspective, it is essential that an enterprise knows the type of knowledge that they need to<br />

focus <strong>on</strong> (Zhao et al., 2012). Using the definiti<strong>on</strong> presented here, the aim of this study is to<br />

understand how software SMEs utilise their knowledge capabilities to achieve their organisati<strong>on</strong>al<br />

goals. It is important to state from the outset that factors such as leadership, culture, people,<br />

organisati<strong>on</strong>al structure, technology and business processes are fundamental to a successful KM<br />

approach (Hasanali, 2002; McDermott and O’Dell, 2001; Storey and Barnett, 2000; Sunassee and<br />

Sewry, 2002) however they were c<strong>on</strong>sidered as part of a larger study and are not the core focus of<br />

this paper. This paper is structured as follows; the subsequent secti<strong>on</strong> briefly outlines the benefits in<br />

pursuing KM. Next a classificati<strong>on</strong> of knowledge activity is defined, the importance of leveraging KM<br />

to deal with envir<strong>on</strong>mental uncertainty is discussed and the research methodology is outlined. In<br />

additi<strong>on</strong>, the background to each case is presented and the findings are c<strong>on</strong>sidered. Finally, the<br />

authors c<strong>on</strong>sider the research findings and present the c<strong>on</strong>clusi<strong>on</strong>s.<br />

2. Harvesting the benefits of KM<br />

Knowledge has become the source of competitive advantage and the source of organisati<strong>on</strong>al<br />

empowerment. N<strong>on</strong>aka (1994) maintains that organisati<strong>on</strong>s must realise the importance of knowledge<br />

in order to survive in a highly competitive market place. He postulates that “in an ec<strong>on</strong>omy where the<br />

<strong>on</strong>ly certainty is uncertainty, the <strong>on</strong>e sure source of lasting competitive advantage is knowledge”<br />

(N<strong>on</strong>aka, 1994, p14). Stewart (1997) further argues that knowledge has become the most important<br />

factor in ec<strong>on</strong>omic life. He acknowledges that knowledge is the chief ingredient in what organisati<strong>on</strong>s<br />

buy and sell, and the raw material with which organisati<strong>on</strong>s work. Intellectual capital, not natural<br />

resources, machinery or even financial capital, has become the <strong>on</strong>e indispensable asset of<br />

corporati<strong>on</strong>s (Stewart, 1997). With due c<strong>on</strong>siderati<strong>on</strong>, the capacity to incorporate and apply the<br />

specialised knowledge of organisati<strong>on</strong>al members is fundamental to a firm’s ability to create and<br />

sustain competitive advantage (Drucker, 1993). This focus has forced organisati<strong>on</strong>s to re-think the<br />

way they manage their business since the emphasis is no l<strong>on</strong>ger <strong>on</strong> tangible assets but <strong>on</strong> people’s<br />

abilities and experiences (Sunassee and Sewry, 2002). As a result organisati<strong>on</strong>s are identifying the<br />

strategies and technologies to manage this knowledge with the objective of gaining maximum benefits<br />

from an organisati<strong>on</strong>'s knowledge pool (O’Dell and Grays<strong>on</strong>, 1998; Sunassee and Sewry, 2002).<br />

However, organisati<strong>on</strong>al knowledge is of limited organisati<strong>on</strong>al value if the knowledge is not shared<br />

and managed (Alavi and Leidner, 1999). Alavi and Leidner (1999) maintain that KM has emerged as a<br />

new philosophy to c<strong>on</strong>trol and support the flow of knowledge in an organisati<strong>on</strong>. In additi<strong>on</strong>, Bansler<br />

and Havn (2002) purport that KM c<strong>on</strong>tributes to improved organisati<strong>on</strong>al productivity, flexibility and<br />

innovati<strong>on</strong> capabilities by enabling employees to share, integrate and reuse knowledge more<br />

effectively. Sunassee and Sewry (2002) further suggest that companies which have implemented KM<br />

soluti<strong>on</strong>s are better equipped to deal with business situati<strong>on</strong>s, as these companies have access to<br />

previous know-how. As a first step, the nature of organisati<strong>on</strong>al knowledge activities (KAs) is<br />

c<strong>on</strong>sidered as a key comp<strong>on</strong>ents of KM.<br />

3. Classificati<strong>on</strong> of knowledge activities<br />

For the purpose of this research, a definiti<strong>on</strong> of KA proposed by Kraaijenbrink et al., (2006, p23) is<br />

adopted as “transacti<strong>on</strong>s or manipulati<strong>on</strong>s of knowledge where the knowledge is the object not the<br />

result”. It is evident that multiple researchers use different terms for the same/similar activity. Many of<br />

these definiti<strong>on</strong>s share comm<strong>on</strong> verbs such as storing, creating and applying knowledge in an<br />

organisati<strong>on</strong>al c<strong>on</strong>text. This research takes a balanced view of KAs, discounting the activities<br />

proposed by Le<strong>on</strong>ard-Bart<strong>on</strong> (1995), as they have a sole technical focus. This research summarises<br />

the terms widely used to describe KAs including knowledge acquisiti<strong>on</strong> (Alavi and Leidner, 2001;<br />

Huber, 1990; Holsapple and Joshi, 2004; Kraaijenbrink et al., 2006), codificati<strong>on</strong> (Davenport and<br />

Prusak, 1998; Faran et al., 2006; Kraaijenbrink et al., 2006; Nevo et al., 2007), storage (Alavi and<br />

Leidner, 2001; Huber, 1991), maintenance (C<strong>on</strong>way and Sligar, 2002; Holsapple and Singh, 2004;<br />

Holsapple and Whinst<strong>on</strong>, 1996), transfer (Alavi and Leidner, 2001; Huber, 1990; N<strong>on</strong>aka and<br />

Takeuchi, 1995) and creati<strong>on</strong> (Davenport and Prusak, 1998; Kayworth and Leidner, 2004; N<strong>on</strong>aka<br />

and Takeuchi, 1995; Pentland, 1995).<br />

118


Ciara Heavin and Frederic Adam<br />

4. Developing knowledge capabilities to deal with uncertainty<br />

Organisati<strong>on</strong>s are cognitive in nature; as a result they learn and develop knowledge (Argyris and<br />

Schön, 1978). Hedberg (1981) defines organisati<strong>on</strong>al learning as a two pr<strong>on</strong>ged process, the first<br />

where organisati<strong>on</strong>s adjust themselves to deal with reality and the sec<strong>on</strong>d where they effectively<br />

leverage knowledge to improve their fit with the external envir<strong>on</strong>ment. In order to achieve this “the<br />

organisati<strong>on</strong> must have mechanisms to learn about and interpret external events” (Daft and Lengel,<br />

1986, p566). In order to maintain and develop organisati<strong>on</strong>al memory it is essential that an<br />

organisati<strong>on</strong> learn from both its internal c<strong>on</strong>text and external envir<strong>on</strong>ment (Bennet and Bennet, 2004).<br />

“Organisati<strong>on</strong>s have no other brains and senses than those of their members” (Hedberg, 1981, p6).<br />

C<strong>on</strong>sidering this perspective, an organisati<strong>on</strong> as an entity is completely reliant <strong>on</strong> the quality and<br />

expertise of the sum of its employees. However, Argyris and Schön (1978) suggest that<br />

organisati<strong>on</strong>s often know less than the sum of their members. This may be due to communicati<strong>on</strong><br />

issues e.g. informati<strong>on</strong> filtering, distorti<strong>on</strong> and channel overload (Argyris and Schön, 1978). The lack<br />

of a formal learning/knowledge repository can c<strong>on</strong>tribute to this. Huber (1989) points out that if<br />

knowledge is not formally stored, it may be lost <strong>on</strong> three counts, firstly through staff turnover, sec<strong>on</strong>dly<br />

through an organisati<strong>on</strong> not knowing what to store based <strong>on</strong> future needs and finally through an<br />

inability to share knowledge. One example of the benefits that may be derived from maintaining a<br />

knowledge repository is Chrysler automobile company. They used an “Engineering Books of<br />

Knowledge” to store an “electr<strong>on</strong>ic memory” of engineer’s past experiences (Davenport and Prusak,<br />

1998). This repository was actively leveraged to inform engineer decisi<strong>on</strong> making in future<br />

development projects.<br />

In terms of problem solving, organisati<strong>on</strong>s build an advantage in boom times however, slack reducti<strong>on</strong><br />

acts as an envir<strong>on</strong>mental indicator of crisis which can activate problem solving mechanisms, and this<br />

in turn can lead to organisati<strong>on</strong>al learning (Hedberg, 1981). In additi<strong>on</strong>, Pounds (1969) c<strong>on</strong>siders<br />

management learning through ‘problem finding’. Where Hedberg (1981) presents slack as a trigger,<br />

Pounds (1969) suggests that problems can be triggered through discrepancies in historical models.<br />

These models act as an archive of past experience to estimate the short term future, though Pounds<br />

(1969) admits in some cases that these models were carried in the heads of management supported<br />

by routine reports. However, opportunity triggers are less evident as problem triggers are more<br />

comm<strong>on</strong> (Hedberg, 1981). Yet, organisati<strong>on</strong>s may identify new opportunities in the market, which in<br />

turn, facilitate learning. Together KM, organisati<strong>on</strong>al learning and memory influence how<br />

organisati<strong>on</strong>s deal with knowledge and its impact <strong>on</strong> organisati<strong>on</strong>al effectiveness (Jennex and<br />

Olfman, 2002).<br />

5. Research approach<br />

This study pursued a qualitative analytical approach (Ägerfalk and Fitzgerald, 2008) using multiple<br />

case studies, each case was selected using purposeful sampling (Patt<strong>on</strong>, 1990). The cases were<br />

selected based <strong>on</strong> their size and industry sector. The software industry “is a knowledge industry. Its<br />

major product is knowledge itself and its major output is research which translates into new products<br />

and services” (p562) (Bernroider, 2002) Software development may be characterised as knowledge<br />

work (Schönstrom and Carlss<strong>on</strong>, 2003). As the objective of this study was to explore the knowledge<br />

approach leveraged by small software development firms, the focus of the study was <strong>on</strong> the two core<br />

business processes of sales and software development. Based <strong>on</strong> a selecti<strong>on</strong> strategy, positi<strong>on</strong>al<br />

methods were used to uncover sales and technical managers while other resp<strong>on</strong>dents were selected<br />

based <strong>on</strong> reputati<strong>on</strong> (Knoke, 1994). Twenty two individuals were interviewed; each interview was<br />

approximately <strong>on</strong>e hour in durati<strong>on</strong>. Interviews were taped and transcribed. The exploratory nature of<br />

the study coupled with the “thick transcripts” (p56) (Miles and Huberman, 1994) meant that qualitative<br />

analysis could be c<strong>on</strong>ducted through the use of coding techniques (Miles and Huberman, 1994). The<br />

classificati<strong>on</strong> of KAs was used for the purpose of data analysis in this study. Each KA was assigned a<br />

code and this code was utilised to classify the nature of KAs, these categories were then assigned<br />

chunks of data derived from the interview transcripts. Each transcript was analysed using the KA<br />

codes and a memo was generated at the level of the interview.<br />

119


6. Findings<br />

6.1 Background to cases<br />

6.2 KM approach at HelpRead Ltd<br />

Ciara Heavin and Frederic Adam<br />

HelpRead was focused <strong>on</strong> building a collective organisati<strong>on</strong>al memory that facilitates c<strong>on</strong>tinued<br />

growth through the introducti<strong>on</strong> of new hires and new products. This is particularly important to them<br />

in terms of acquiring external knowledge to inform new product development. Table 1 identifies 82<br />

instance of KA at HelpRead; the majority of activities presented themselves through knowledge<br />

acquisiti<strong>on</strong>, storage and transfer.<br />

Table 1: Distributi<strong>on</strong> of KAs at HelpRead Ltd<br />

The study identified 82 KAs; 113 instances of knowledge types were uncovered. The difference in<br />

these figures can be explained by single KAs leveraging multiple knowledge types in some instances,<br />

thus increasing the instances of knowledge types identified in the analytic memos. It is also important<br />

to note that, at the time, HelpRead Ltd. was not in a new product development phase. Using Table 3,<br />

the difference in intensity between these types of activities is indicative of HelpRead’s positi<strong>on</strong> as a<br />

growing organisati<strong>on</strong>. Knowledge acquisiti<strong>on</strong> intensity at 21 percent (n=82) showed that fifty three<br />

120


Ciara Heavin and Frederic Adam<br />

percent of all knowledge acquisiti<strong>on</strong> activity was focused <strong>on</strong> gathering product knowledge supporting<br />

Groen’s (2006, p124) view that in high-technology SMEs require knowledge from external sources to<br />

support new product development.<br />

At 13 percent (n=82) codificati<strong>on</strong> activity was relatively low intensity, this was reflective of the<br />

uncertainty around what the company needs to know in the future. This is predominantly evident with<br />

the Technical FAQ, which lacked buy-in from the entire development team. The Development<br />

Manager admitted that as a team “they didn’t know what they should know”. Most codificati<strong>on</strong> activity<br />

was directly related to refining the discussi<strong>on</strong>s at group meetings into documents which are made<br />

available over the Intranet. Over 90 percent (n=11) of all codificati<strong>on</strong> activity identified in Table 1 was<br />

related to product development knowledge. Codificati<strong>on</strong> was largely not a sales related activity. The<br />

well-defined scope of the Goldmine TM sales system meant that no KA was required to support the<br />

refinement and distillati<strong>on</strong> of sales related knowledge. In additi<strong>on</strong>, the experience of the sales team<br />

meant they know what important customer and sales related knowledge should be stored for future<br />

use.<br />

The high occurrence of storage activities at 29 percent was indicative of the importance placed <strong>on</strong><br />

storing knowledge in the new Intranet-based quality system - approx 74 percent (17 of n=24 storage<br />

activities) of storage activity involved the Intranet. These activities primarily included storing software<br />

project documents and employee skills documents, in line with the compliance requirements outlined<br />

by IS9001:2000. The codificati<strong>on</strong> intensity also included the level of customer informati<strong>on</strong> captured<br />

and stored by the sales team. This 29 percent reflected the move to store the knowledge gathered<br />

from acquisiti<strong>on</strong>, codificati<strong>on</strong> and transfer activities. Maintenance activities at 10 percent highlighted<br />

the company’s focus <strong>on</strong> maintaining software and product development knowledge. Surprisingly,<br />

transfer activity was high intensity at 19 percent - with closer inspecti<strong>on</strong>; the role of the Technical<br />

Director was integral to this. At 6 percent, knowledge creati<strong>on</strong> was very low. While Table 1 shows that<br />

80 percent of knowledge creati<strong>on</strong> activity was focused <strong>on</strong> product knowledge, in line with company<br />

strategy, the lack of other types of knowledge creati<strong>on</strong> may be explained by the pressures associated<br />

with the recent rapid growth in employee headcount and the increased product portfolio.<br />

6.3 KM approach at TravelSoft Ltd<br />

At the time of interview a new Applicati<strong>on</strong> Soluti<strong>on</strong>s Manager had been in place at TravelSoft for<br />

approximately eight m<strong>on</strong>ths. From a Telecoms background, he implemented a number of<br />

organisati<strong>on</strong>al strategies to develop embedded processes and most importantly to bring a new<br />

product to the travel software marketplace. It is primarily these management initiatives that<br />

c<strong>on</strong>tributed to the high number of KAs, 147 instances presented in Table 2.<br />

Table 2: KAs at TravelSoft Ltd<br />

The KAs identified used more than <strong>on</strong>e type of knowledge during a single KA, c<strong>on</strong>sequently providing<br />

the rati<strong>on</strong>ale for the 211 instances of knowledge types identified for TravelSoft. The knowledge focus<br />

121


Ciara Heavin and Frederic Adam<br />

at TravelSoft was quite c<strong>on</strong>sistent and reflected the company’s strategic objectives. The emphasis <strong>on</strong><br />

software development, project, process and product knowledge was marked. Knowledge of the travel<br />

industry made up a quarter of the knowledge acquisiti<strong>on</strong> activity.<br />

At HelpRead Ltd. 82 instances of KAs were observed, KA at TravelSoft was c<strong>on</strong>siderably higher at<br />

147 instances. This intensity may be explained by a number of factors. Knowledge acquisiti<strong>on</strong> activity<br />

at 11 percent (16 of n=147) was due to the acquisiti<strong>on</strong> of c<strong>on</strong>sultant knowledge <strong>on</strong> new product<br />

development, employee training, relevant books, journals and travel c<strong>on</strong>ferences. In terms of<br />

codificati<strong>on</strong> at 20 percent activity, project related knowledge was refined and stored. At 21 percent<br />

activity, storage activity was almost in line with the volume of codificati<strong>on</strong> activity. This shows that<br />

TravelSoft were good at following through <strong>on</strong> this type of activity. For example the steering committee<br />

refine and store the new Adept framework templates in the relevant artefacts. R&D acquisiti<strong>on</strong>,<br />

codificati<strong>on</strong>, storage and maintenance of knowledge c<strong>on</strong>tribute to the dense volumes of KA. Activities<br />

such as Internet research in the travel area added to the level of knowledge acquisiti<strong>on</strong> activities,<br />

while refining and storing this knowledge c<strong>on</strong>tributed to the volume of codificati<strong>on</strong> and storage activity.<br />

At 14 percent, maintenance activity was lower than knowledge codificati<strong>on</strong> and storage activity. This<br />

could be owing to the fact that some of the knowledge stored did not require updating, for example<br />

c<strong>on</strong>ference and journal papers <strong>on</strong> the travel industry will not be changed although new papers may be<br />

added over time resulting in increased storage activity.<br />

Knowledge transfer at 28 percent (41 where n=147) represented the highest volume of KA.<br />

Leveraging a variety of routine and n<strong>on</strong>-routine modes (these are outlined in next secti<strong>on</strong>). This<br />

organisati<strong>on</strong> encouraged knowledge transfer at all levels of the organisati<strong>on</strong>. Knowledge creati<strong>on</strong> was<br />

much lower at 9 instances (6 percent where n=147). These activities were all generated around new<br />

product and process development placing these initiatives at the core of all KAs of TravelSoft at that<br />

time. Table 2 shows a spread of 66 percent of KA at TravelSoft across knowledge acquisiti<strong>on</strong>,<br />

codificati<strong>on</strong>, storage and maintenance activity, while transfer and creati<strong>on</strong> activity account for 34<br />

percent of all KA. By comparis<strong>on</strong>, the distributi<strong>on</strong> at HelpRead for the same activities was 73 percent<br />

and 27 percent respectively. This shows that through their change process, TravelSoft were good at<br />

leveraging the more ‘valuable’ types of KA.<br />

6.4 KM approach at Systems Soluti<strong>on</strong>s Ltd<br />

KA at Systems Soluti<strong>on</strong>s was mostly characterised by its informal nature. The Managing Director<br />

admitted that when he is involved with requirements analysis for the business intelligence divisi<strong>on</strong> the<br />

knowledge is documented and stored in an A4 pad. One Project Manager from the Applicati<strong>on</strong><br />

Divisi<strong>on</strong> admitted that it was not uncomm<strong>on</strong> to calculate a project price <strong>on</strong> the back of a piece of<br />

paper in the car park before attending a meeting with a prospective customer. Table 3 illustrates a<br />

total of 105 KAs identified.<br />

Table 3: KAs at Systems Soluti<strong>on</strong>s<br />

One hundred and thirty <strong>on</strong>e instances of knowledge type were identified across the KAs; this indicates<br />

that some KAs leveraged multiple knowledge types. It is apparent that knowledge acquisiti<strong>on</strong> and<br />

maintenance were excepti<strong>on</strong>s in terms of their knowledge focus. Knowledge acquisiti<strong>on</strong> was focused<br />

122


Ciara Heavin and Frederic Adam<br />

<strong>on</strong> product and customer knowledge, these knowledge types were largely relevant to the Business<br />

Service <strong>Management</strong> and SAP Soluti<strong>on</strong>s divisi<strong>on</strong>s focused <strong>on</strong> software resale. Knowledge<br />

maintenance activity was focused <strong>on</strong> sales knowledge at 38 percent. The emphasis <strong>on</strong> sales primarily<br />

reflects the knowledge requirements of these two divisi<strong>on</strong>s. From Table 3, the other KAs were<br />

focused <strong>on</strong> software development and project knowledge serving the knowledge needs of the<br />

Business Intelligence (data warehousing) and Applicati<strong>on</strong> Development divisi<strong>on</strong>s.<br />

Project related knowledge was codified, stored and maintained in order to meet the requirements of<br />

pharmaceutical customers who must abide by Food and Drugs Authority (FDA) regulati<strong>on</strong>s. From<br />

Table 3, it is evident that at 26 percent, storage activity was higher than both codificati<strong>on</strong> and<br />

maintenance activity together at 21 percent. This may mean that Systems Soluti<strong>on</strong>s store large<br />

volumes of documentati<strong>on</strong> without refining and formatting it, and in the l<strong>on</strong>ger term, without updating<br />

it. As a result, it seemed that they hold large these volumes for the sole purpose of protecting<br />

themselves from external threats such as possible audits. At 15 percent, knowledge acquisiti<strong>on</strong><br />

appeared important, however more than half of this activity is attributed to sales and customer<br />

interacti<strong>on</strong>.<br />

At 5 percent, knowledge creati<strong>on</strong> activity was very low. The Managing Director was the main source<br />

of the knowledge creati<strong>on</strong> activity at Systems Soluti<strong>on</strong>s. It seems that the time pressures associated<br />

with meeting project deadlines meant that there was little time for knowledge creati<strong>on</strong> activity am<strong>on</strong>gst<br />

the divisi<strong>on</strong>s. In the case of Systems Soluti<strong>on</strong>s knowledge creati<strong>on</strong> was not the resp<strong>on</strong>sibility of those<br />

at an operati<strong>on</strong>al level.<br />

Knowledge acquisiti<strong>on</strong>, codificati<strong>on</strong>, storage and maintenance account for 66 percent of all KAs while<br />

knowledge transfer and creati<strong>on</strong> amount to 34 percent. This was c<strong>on</strong>sistent with TravelSoft though it<br />

differed in the case of HelpRead whose focus <strong>on</strong> knowledge storage activity through the new<br />

company Intranet tips the balance of KA distributi<strong>on</strong> towards the earlier activities.<br />

6.5 KM approach at DocMan (Ireland) Ltd<br />

At DocMan (Ireland) the total volume of KA was low in comparis<strong>on</strong> to the other cases c<strong>on</strong>sidered.<br />

This may be endorsed by the nature of the well-defined work <strong>on</strong> software development comp<strong>on</strong>ents at<br />

the DocMan site in Ireland. The operati<strong>on</strong>s at the Irish site are part of a larger document management<br />

software comp<strong>on</strong>ent and the output from DocMan (Ireland) was integrated by the software integrator<br />

at the Swiss headquarters. As a result of this task specificity, there was a set of core KAs from which<br />

there was minimal diversificati<strong>on</strong> at the Irish site. The breakdown of KAs for DocMan (Ireland) Ltd. is<br />

presented in Table 4.<br />

Table 4: KAs at DocMan (Ireland) Ltd<br />

Table 4 illustrates a significant level of knowledge c<strong>on</strong>sistency across all of the KAs. Software<br />

development and project knowledge represented at least 57 percent of the knowledge focus for all six<br />

KAs. This uniformity across activities also supports the task specialisati<strong>on</strong> activity at the DocMan<br />

(Ireland) site.<br />

DocMan (Ireland) leveraged some external knowledge resources at 12 percent (7 where n=60)<br />

knowledge acquisiti<strong>on</strong>, however the main source of knowledge is the headquarters in Switzerland and<br />

this was achieved through knowledge transfer activity which was very high at 40 percent (24 where<br />

123


Ciara Heavin and Frederic Adam<br />

n=60) of total activity. It is from here that the majority of customer requirements were received as well<br />

as any new product knowledge.<br />

Knowledge creati<strong>on</strong> activity was very low at 3 percent (2 where n=60). This may be the result of the<br />

locati<strong>on</strong> of this development team with most new ideas being generated at a higher level at company<br />

headquarters.<br />

Although the total volume of KA was low, the split between the acquire, codify, store and maintain KA<br />

at 57 percent and the knowledge transfer and creati<strong>on</strong> activity at 43 percent is more evenly balanced<br />

than that observed at HelpRead Ltd., at 73 percent to 27 percent respectively. The geographic<br />

locati<strong>on</strong>, the task specialisati<strong>on</strong> and the maturity of the parent organisati<strong>on</strong> may be attributed to the<br />

knowledge transfer capabilities identified at DocMan Ireland.<br />

7. Discussi<strong>on</strong><br />

The previous secti<strong>on</strong> explores the distributi<strong>on</strong> of KAs across four software SMEs. As part of this<br />

c<strong>on</strong>siderati<strong>on</strong>, it is imperative to take a closer look at the firm’s motivati<strong>on</strong> for pursuing different KAs in<br />

terms of their wider organisati<strong>on</strong>al objectives. This is even more crucial in the case of an SME where<br />

their ability to leverage the resources available to be them, enabling them to be flexible enough to<br />

pursue alternative organisati<strong>on</strong>al goals is essential, as they are more susceptible to external forces.<br />

Table 5 provides a sample of the organisati<strong>on</strong>al goals pursued by the SMEs. Each goal is<br />

characterised by the knowledge types and KAs used to achieve the objectives of the firm at a<br />

particular time.<br />

Table 5: Achieving organisati<strong>on</strong>al goals through knowledge activity<br />

Table 5 highlights that the KM approaches differ across organisati<strong>on</strong>s; this is typically due to the<br />

difference in culture, organisati<strong>on</strong>al structure and organisati<strong>on</strong>al objectives therefore organisati<strong>on</strong>s<br />

need to identify a unique strategy that suits their needs. Hansen et al. (1999, p109) state that, “a<br />

company's knowledge management strategy should reflect its competitive strategy” and this is widely<br />

reinforced in extant literature (Davenport and Prusak, 1998; Hasanali, 2002; Sunassee and Sewry,<br />

2002). Aligning KM strategy to the business strategy seeks to clarify what the company must know to<br />

in order to realise what the company can do. It is well supported that organisati<strong>on</strong>s, which did not<br />

develop a separate KM strategy, ended up with their KM initiative losing focus, priority and impact<br />

(Chourides et al., 2003).C<strong>on</strong>sidering the lens (KAs) used in this study to characterise the firms KM<br />

approach, Figure 2 below provides a holistic view of the key comp<strong>on</strong>ents examined, these include<br />

knowledge type, KAs, the underlying organisati<strong>on</strong>al strategy and the benefits derived from the KM<br />

approach. Figure 2. illustrates a diagnostic instrument which could provide SMEs with the capability<br />

of tangibly measuring their current KM approach.<br />

Ideally, in times of uncertainty the firm should be flexible enough to leverage knowledge capabilities in<br />

order to pursue the goals of the organisati<strong>on</strong> at that time. This formalised systematic approach may<br />

result in the establishment of knowledge rules which can be followed enabling the firm to develop<br />

embedded KAs. By doing this, the organisati<strong>on</strong> can learn from past experience to inform future<br />

development.<br />

124


Figure 2: An overview of the KM approach<br />

8. C<strong>on</strong>clusi<strong>on</strong><br />

Ciara Heavin and Frederic Adam<br />

KM may be achieved if it is closely aligned with the strategic needs of the organizati<strong>on</strong>. This approach<br />

seeks to identify the organizati<strong>on</strong>’s requirements and evaluate a knowledge strategy based <strong>on</strong> the<br />

business’s strategic visi<strong>on</strong>. In a software development company, <strong>on</strong>e organisati<strong>on</strong>al objective may be<br />

to improve the efficiency of the software developers as a means of increasing profits <strong>on</strong> individual<br />

projects. As a result an organisati<strong>on</strong>’s knowledge strategy may be to ‘avoid reinventing the wheel’<br />

leveraging existing programming code in new projects. Ideally the appropriate knowledge capabilities<br />

should be in place to resp<strong>on</strong>d to the changing objectives of the firm or even support multiple goals<br />

e.g. to support new product development activity and a focused sales strategy to improve customer<br />

relati<strong>on</strong>ship management. SMEs need to give formal c<strong>on</strong>siderati<strong>on</strong> to their KM approach, in order to<br />

manipulate knowledge in a way that serves their specific decisi<strong>on</strong> making needs at a particular time.<br />

References<br />

Ägerfalk, P. J. and Fitzgerald, B. (2008) Outsourcing to an Unknown Workforce: Exploring Opensourcing as a<br />

Global Sourcing Strategy, MIS Quarterly, Vol. 32. No.2, pp385-409.<br />

Alavi, M. & Leidner, D.E. (2001) Knowledge <strong>Management</strong> and Knowledge <strong>Management</strong> Systems: C<strong>on</strong>ceptual<br />

Foundati<strong>on</strong>s and Research Issues, MIS Quarterly, 25(1), pp. 107-136.<br />

Argyris, C. and Schön, D.A., (1978) Organizati<strong>on</strong>al Learning: A Theory of Acti<strong>on</strong> Perspective. Addis<strong>on</strong> Wiley,<br />

Reading, MA.<br />

Bansler, J.P. and Havn, E. (2002) “Exploring the Role of Network Effects in IT Implementati<strong>on</strong>: The Case of the<br />

Knowledge <strong>Management</strong> Systems”, 10 th <str<strong>on</strong>g>European</str<strong>on</strong>g> <str<strong>on</strong>g>C<strong>on</strong>ference</str<strong>on</strong>g> <strong>on</strong> Informati<strong>on</strong> Systems, June 6-8, pp. 817-<br />

829.<br />

Bennet, A. and Bennet, D. (2004) “The Rise of the Knowledge Organizati<strong>on</strong>”, The Handbook <strong>on</strong> Knowledge<br />

<strong>Management</strong> Volume 1, (Eds). C.W. Holsapple, Springer, Germany, pp. 1-20.<br />

Bernroider, E. (2002) “Factors in SWOT Analysis Applied to Micro, Small-to-Medium, and Large Software<br />

Enterprises: An Austrian Study”, <str<strong>on</strong>g>European</str<strong>on</strong>g> <strong>Management</strong> Journal, Vol. 20, No. 5, pp. 562-573.<br />

Chourides, P., L<strong>on</strong>gBottom, D. and Murphy, W. (2003) “Excellence in knowledge management: an empirical<br />

study to identify critical factors and performance measures”, Measuring Business Excellence, Vol. 7, No.2,<br />

pp. 29-45.<br />

C<strong>on</strong>way, S. and Sligar, C. (2002) Unlocking Knowledge Assets, Microsoft Press, Redm<strong>on</strong>d, WA.<br />

Daft, R. L. and Lengel, R.H. (1986) ‘Organizati<strong>on</strong>al informati<strong>on</strong> requirements, media richness and structural<br />

design’, <strong>Management</strong> Science, Vol. 32, No. 5, pp. 554-571.<br />

Davenport, T.H., and Prusak, L. (1998) Working Knowledge. How Organizati<strong>on</strong>s Manage What They Know.<br />

Bost<strong>on</strong>, Mass.: Harvard Business School Press, 1998. Transacti<strong>on</strong> Engineering <strong>Management</strong>, 23, pp163-<br />

167.<br />

125


Ciara Heavin and Frederic Adam<br />

Drucker, P. (1993) The Post-Capitalist Society, Butterworth-Heinemann, Oxford.<br />

Faran, D. (2006) “Assessment – Making Sense of it all”, Knowledge Integrati<strong>on</strong>: The Practice of Knowledge<br />

<strong>Management</strong> in Small and Medium Sized Enterprises, Jetter, A, Kraaijenbrink, J. and Wijnhoven, F. (eds),<br />

pp. 101-114.<br />

Hansen, M., Nohria, N and Tierney, T. (1999): What’s Your Strategy for Managing Knowledge? Harvard Business<br />

Review, March-April, 106-116.<br />

Hasanali, F. (2002) “Critical Success Factors of Knowledge <strong>Management</strong>”, Available<br />

At:http://www.providersedge.com/docs/km_articles/Critical_Success_Factors_of_KM.pdf (Last Accessed<br />

June 14 th 2009)<br />

Hedberg, B. (1981) “How organisati<strong>on</strong>s learn and unlearn”, Nystrom and Starbuck (Eds) Handbook of<br />

Organisati<strong>on</strong>al Design, Vol. 2, Oxford University Press, England, pp. 3-27.<br />

Holsapple, C.W. and Whinst<strong>on</strong>, T. (1987) Knowledge-based orgnanizati<strong>on</strong>s, The Informati<strong>on</strong> Society, 2, pp77-<br />

90.<br />

Holsapple, C. and Singh, M. (2004) The Knowledge Chain Model: Activities for Competitiveness, Handbook <strong>on</strong><br />

Knowledge <strong>Management</strong>, Eds Holsapple, C.W. ,Verlanger: Berlin.<br />

Holsapple, C. and Joshi, K. (2004) A Knowledge <strong>Management</strong> Ontology, Handbook <strong>on</strong> Knowledge <strong>Management</strong>,<br />

Eds Holsapple, C.W. ,Verlanger: Berlin.<br />

Huber, G.P. (1984) “The Nature and Design of Post Industrial Organisati<strong>on</strong>s”, <strong>Management</strong> Science, Vol. 30, No.<br />

8, pp. 928-951.<br />

Huber, G.P. (1989) “A Theory of the Effects of Advanced Informati<strong>on</strong> Technologies <strong>on</strong> Organizati<strong>on</strong>al Design,<br />

Intelligence, and Decisi<strong>on</strong> Making”, Organizati<strong>on</strong>s and Communicati<strong>on</strong> Technology, pp. 237-274.<br />

Huber, G.P. (1990). A theory of the effects of advanced informati<strong>on</strong> technologies <strong>on</strong> organizati<strong>on</strong>al design,<br />

intelligence, and decisi<strong>on</strong> making, <strong>Academic</strong> <strong>Management</strong> Review, 15(1), pp47-71.<br />

Huber, G.P. (1991), Organisati<strong>on</strong>al Learning: The c<strong>on</strong>tributing Processes and the Literatures, Organisati<strong>on</strong><br />

Science, Vol. 2, No.1, pp88-115.<br />

Jennex, M. E. and Olfman, L. (2002) “Organizati<strong>on</strong>al Memory/Knowledge Effects <strong>on</strong> Productivity”, A L<strong>on</strong>gitudinal<br />

Study, HICSS35, IEEE Computer Society.<br />

Kayworth, T. and Leidner, D. (2004) Organizati<strong>on</strong>al Culture as a Knowledge Resource, Handbook of Knowledge<br />

<strong>Management</strong> 1, Eds Holsapple C.W., Verlanger: Berlin.<br />

Knoke, D. (1994) Networks of elite structure and decisi<strong>on</strong> making, in Advances in Network Analysis: Research in<br />

the Social and Behavioural Sciences, eds. S. Wasserman and J. Galaskiewicz, Sage, Thousand Oaks, CA.<br />

Kraaijenbrink, J., Faran, D. and Hauptman, A. (2006).Knowledge Integrati<strong>on</strong> by SMEs – Framework, Knowledge<br />

Integrati<strong>on</strong>: The Practice of Knowledge <strong>Management</strong> in Small to Medium Sized Enterprises. Eds Jetter, A.,<br />

Kraaijenbrink, J. Schroder, H., Wijnhoven, F., Springer.<br />

Le<strong>on</strong>ard-Bart<strong>on</strong>, D. (1995) Wellsprings of knowledge: building and sustaining the sources of innovati<strong>on</strong>, Bost<strong>on</strong> ,<br />

MA: Harvard Business School Press.<br />

McDermott, R. and O’Dell, C. (2001) “Overcoming Cultural Barriers to Sharing Knowledge”, Journal of<br />

Knowledge <strong>Management</strong>, Vol. 5, No. 1, pp. 76-85.<br />

Miles, M.B and Huberman, A.M. (1994) Qualitative Data Analysis, Sage Publicati<strong>on</strong>s.<br />

Nevo, S., Wade, M.R., Cook, W.D. (2007). An examinati<strong>on</strong> of the trade off between internal and external IT<br />

capabilities, Journal of Strategic Informati<strong>on</strong> Systems, 16, pp5-23.<br />

N<strong>on</strong>aka, I. (1994) “A Dynamic Theory of Organizati<strong>on</strong>al Knowledge Creati<strong>on</strong>”, Organizati<strong>on</strong> Science, Vol. 5, pp.<br />

14-37.<br />

N<strong>on</strong>aka, I. and Takeuchi, H. (1995) The Knowledge-Creating Company. Oxford University Press, New York, NY.<br />

O'Dell, C., Grays<strong>on</strong>, C.J.J. (1998) If Only We Knew What We Know: The Transfer of Internal Knowledge and<br />

Best Practice, The Free Press, New York, NY.<br />

Patt<strong>on</strong>, M. Q. (1990) Qualitative Evaluati<strong>on</strong> and Research Methods, Sage Publicati<strong>on</strong>s, Thousand Oaks,<br />

California.<br />

Pentland, B. (1995). “Informati<strong>on</strong> systems and organizati<strong>on</strong>al learning: the social epistemology of organizati<strong>on</strong>al<br />

knowledge systems”. Accounting, <strong>Management</strong> and Informati<strong>on</strong> Technologies, Vol. 5 No. 1, pp1-21.<br />

Pounds, W. F. (1969) “The Process of Problem Finding”, IMR, Fall, pp. 1-19<br />

Steward, T.A. (1997) Intellectual Capital: The New Wealth of Organisati<strong>on</strong>s, Nicholas Brealy, L<strong>on</strong>d<strong>on</strong>.<br />

Schönstrom, M. and Carlss<strong>on</strong>, S, A. (2003) “Methods as Knowledge Enablers in Software Development<br />

Organizati<strong>on</strong>s“, In Proceedings of the Eleventh <str<strong>on</strong>g>European</str<strong>on</strong>g> <str<strong>on</strong>g>C<strong>on</strong>ference</str<strong>on</strong>g> <strong>on</strong> Informati<strong>on</strong> Systems, pp. 1707-<br />

1718.<br />

Storey, J. and Barnett, E. (2001) “Knowledge <strong>Management</strong> initiatives: learning from failure”, Journal of<br />

Knowledge <strong>Management</strong>, Vol. 4, No. 2, pp.145-146.<br />

Sunassee, N. N. and Sewry, D. A. (2002)”A theoretical Framework for knowledge management implementati<strong>on</strong>”,<br />

Proceedings of SAICSIT 2002, pp. 235-245.<br />

Szulanski, G. (1994) Intra-Firm Transfer of Best Practices Project. American Productivity and Quality Center.<br />

Houst<strong>on</strong>, Texas.<br />

Wurman, R. S. (2001) Informati<strong>on</strong> Anxiety 2. Que, Indiana, USA.<br />

Zhao, J.., Ordóñez de Pablos, P. and Qi, Z. (2012) “Enterprise knowledge management model based <strong>on</strong> China’s<br />

practice and case study”, Computers in Human Behavior, 28, pp. 324–330.<br />

126


Analyzing Less<strong>on</strong>s Learned to Identify Potential Risks in<br />

new Product Development Projects<br />

Vered Holzmann<br />

Faculty of <strong>Management</strong> of Technology, Hol<strong>on</strong> institute of Technology – H.I.T,<br />

Hol<strong>on</strong>, Israel<br />

veredhz@hit.ac.il<br />

Abstract: The paper presents a methodological implementati<strong>on</strong> of the synergetic relati<strong>on</strong> between past and<br />

future by utilizing analysis of past occurrences to mitigate future possible risks. When an organizati<strong>on</strong> undertakes<br />

the development of a new product, it should prepare a risk management plan that can outline the guidelines to<br />

mitigate possible uncertainties with a negative impact <strong>on</strong> the development project or the expected product. This<br />

process can be performed by analyzing the informati<strong>on</strong> accumulated during previous projects executed by the<br />

organizati<strong>on</strong> during the processes of developing other deliverables. The method of transforming organizati<strong>on</strong>al<br />

informati<strong>on</strong> into applicable risk management guidelines is based <strong>on</strong> two research techniques: c<strong>on</strong>tent analysis<br />

and cluster analysis. The c<strong>on</strong>tent analysis approach requires a deep understanding of the less<strong>on</strong>s learned<br />

dataset collected from past projects while documenting undesired events or failures. This process integrates<br />

qualitative and quantitative procedures including a review of each record, a classificati<strong>on</strong> of relevant hazardous<br />

factors, and a computati<strong>on</strong> of recurring factors in completed projects and processes. The cluster analysis<br />

approach uses the risk dataset to create a risk tree that represents relative weights for each risk factor, while<br />

c<strong>on</strong>sidering the reoccurrence of similar events in similar circumstances. Implementati<strong>on</strong> of the methodology in<br />

order to identify potential risks, in particular analyzing less<strong>on</strong>s learned in technological organizati<strong>on</strong>s, yielded<br />

interesting organizati<strong>on</strong>al risk tree that shows a substantial weight accumulati<strong>on</strong> in the areas of<br />

miscommunicati<strong>on</strong>s and misunderstanding of stakeholder resp<strong>on</strong>sibilities. The results emphasized the following<br />

susceptible items: resp<strong>on</strong>sibility definiti<strong>on</strong>, delivery method, communicati<strong>on</strong> and informati<strong>on</strong> needs and<br />

resp<strong>on</strong>sibilities, and change management. The findings exposed the 'soft skills' of project managers and project<br />

teams, rather than technical issues or engineering problems, as being the vulnerable areas that should be<br />

managed carefully in order to finish the project successfully. The study offers a generic validated methodology for<br />

risk identificati<strong>on</strong> based <strong>on</strong> analysis of less<strong>on</strong>s learned, supported by results of an implementati<strong>on</strong> of the<br />

methodology in a high-tech company. The method of analysis can be applied by managers of new product<br />

development projects to identify risk issues, classify them into groups, and c<strong>on</strong>struct a risk tree that represents<br />

the project risk areas and their relative weights.<br />

Keywords: risk management; less<strong>on</strong>s learned; project management; clustering; c<strong>on</strong>tent analysis<br />

1. Introducti<strong>on</strong><br />

Every organizati<strong>on</strong> aims to get a competitive advantage over the other organizati<strong>on</strong>s playing in the<br />

same arena. Especially today, when the business battle field is usually global, very dynamic and is<br />

subject to countless changes, any opportunity to be better prepared for the future creates an<br />

advantage. Therefore, a more accurate and relevant risk management plan will give any organizati<strong>on</strong><br />

this required advantage over its competitors. The current research was targeted to provide a better<br />

and innovative method to set up risk management plans based <strong>on</strong> a systematic methodology<br />

c<strong>on</strong>structed of an array of c<strong>on</strong>secutive sequential steps. The development of the methodology is<br />

based <strong>on</strong> two assumpti<strong>on</strong>s: (1) Modern organizati<strong>on</strong>s manage portfolio of projects. Many of the<br />

organizati<strong>on</strong>s nowadays are operating through a c<strong>on</strong>tinuous implementati<strong>on</strong> of projects. Although<br />

every project is a different endeavor, organizati<strong>on</strong>s are c<strong>on</strong>stantly moving towards achieving their<br />

visi<strong>on</strong> by managing a portfolio of projects. Since all, or at least most, of the organizati<strong>on</strong>al past<br />

projects were performed within a similar envir<strong>on</strong>ment, and usually managed and at least partially<br />

carried out by the same teams, it is worthwhile to embrace a broad approach that takes into account<br />

not <strong>on</strong>ly the individual project but rather the organizati<strong>on</strong>al project portfolio (Kerzner, 2005). (2) Every<br />

organizati<strong>on</strong>’s archive c<strong>on</strong>tains huge volume of historical valuable informati<strong>on</strong>. Almost every modern<br />

organizati<strong>on</strong> possesses an archive that c<strong>on</strong>tains a huge amount of informati<strong>on</strong> that can be transposed<br />

into valuable management knowledge. Modern communicati<strong>on</strong> technologies enable people and<br />

organizati<strong>on</strong>s to create, distribute, receive and store large quantities of data. There are number of<br />

tools and techniques that an organizati<strong>on</strong> might use in order to transform its data into informati<strong>on</strong>, and<br />

further into knowledge. Since the organizati<strong>on</strong> already has this informati<strong>on</strong>, no additi<strong>on</strong>al efforts are<br />

required for its producti<strong>on</strong> (Lipshitz, Popper and Friedman, 2008; Hill, 2008).<br />

Based <strong>on</strong> these two c<strong>on</strong>cepts, the research was targeted to develop a structured methodology for<br />

project oriented organizati<strong>on</strong>s to design the guidelines for a risk management plan based <strong>on</strong> an<br />

127


Vered Holzmann<br />

analysis of documents that are already exist in the organizati<strong>on</strong>. Therefore, the research objectives<br />

were defined as follows:<br />

To develop a generic methodology that can identify organizati<strong>on</strong>al future risks based <strong>on</strong><br />

documented organizati<strong>on</strong>al known history. The methodology focuses <strong>on</strong> representing the<br />

transpositi<strong>on</strong> of existing informati<strong>on</strong>, available in the form of documents or electr<strong>on</strong>ic records,<br />

either structured or unstructured, into valuable managerial knowledge, for the purpose of<br />

identifying prospective events, acti<strong>on</strong>s or occurrences.<br />

To dem<strong>on</strong>strate the generic methodology by formulating a Risk Breakdown Structure (RBS) from<br />

less<strong>on</strong>s learned documents. The study was targeted to generate a specific risk management tool<br />

– an RBS, which is an organizati<strong>on</strong>al risk hierarchy, from a specific type of documents, i.e.,<br />

less<strong>on</strong>s learned documents.<br />

The paper starts with a review of risk management tools and techniques, especially those based <strong>on</strong><br />

documentati<strong>on</strong> analysis methods, followed by a brief descripti<strong>on</strong> of the developed methodology to<br />

transform less<strong>on</strong>s learned into outputs of a risk management plan. Then, a case study in an Israeli<br />

high-tech company is presented. The detailed analysis is presented with a list of identified risks,<br />

categorized by subjects and interrelati<strong>on</strong>s. Final comments summarize the results and the c<strong>on</strong>tributi<strong>on</strong><br />

of the current research to the academic and professi<strong>on</strong>al communities.<br />

2. Risk management tools and techniques<br />

Cohen and Palmer (2004) define risk as “the potential for complicati<strong>on</strong>s and problems with respect to<br />

the completi<strong>on</strong> of a task and the achievement of a goal” (p.IN11). Other definiti<strong>on</strong>s emphasize the<br />

percepti<strong>on</strong> of risks as uncertainties. Thus, suggest a broader interpretati<strong>on</strong> of the term “risk” as<br />

management of opportunities, i.e., positive occurrences, rather than <strong>on</strong>ly management of potential<br />

negative risks (e.g., Ward and Chapman, 2003; Garrett, 2005; Olss<strong>on</strong>, 2007).<br />

2.1 Risk management methodology<br />

The Guide to Project <strong>Management</strong> Body Of Knowledge (PMBOK® Guide), published by the Project<br />

<strong>Management</strong> Institute (PMI, 2008), describes the knowledge area of project risk management as the<br />

processes that address risk management planning, identificati<strong>on</strong>, analysis, resp<strong>on</strong>se, and m<strong>on</strong>itoring<br />

and c<strong>on</strong>trol. Project risk management processes are targeted to increase the probability and impact of<br />

events that are expected to positively affect the project as well as to decrease the probability and<br />

impact of events that are expected to negatively affect the project or the achievement of its objectives.<br />

Effective implementati<strong>on</strong> of a risk management methodology and practice positively affects the<br />

success rate of any project or process. The relati<strong>on</strong>ship between risk management and project<br />

success has been described by various researchers investigating different industries (e.g., Raz,<br />

Shenhar and Dvir, 2002; Cook, 2005; Nalewaik, 2005; Marxt and Link, 2002). The risk management<br />

methodology presented by the PMBOK Guide (PMI, 2008) starts with risk identificati<strong>on</strong>, proceeds<br />

through risk qualitative and quantitative assessment, followed by planning risk resp<strong>on</strong>se, and ends<br />

with a c<strong>on</strong>tinuous process of risk m<strong>on</strong>itoring and c<strong>on</strong>trol. Similar methodology is presented by the SEI<br />

(Software Engineering Institute) (Higuera and Haimes, 1996) and by the ERM (Enterprise Risk<br />

<strong>Management</strong>) framework, developed by the Committee of Sp<strong>on</strong>soring Organizati<strong>on</strong>s of the Treadway<br />

Commissi<strong>on</strong> (COSO, 2004). The PMBOK® Guide methodology can be described by the following<br />

chart.<br />

Figure 1: Risk management methodolog by the PMBOK guide<br />

128


Vered Holzmann<br />

The risk identificati<strong>on</strong> process detects prospective events which might affect the project and<br />

documents their characteristics. Risk assessment deals with evaluating two fundamental parameters<br />

with regard to each identified risk event: the probability of occurrence and the impact of the identified<br />

risk events. The risk evaluati<strong>on</strong> might be either qualitative or quantitative, and it is used to grade each<br />

of the identified risk events in order to prioritize them. Risk resp<strong>on</strong>se planning involves the<br />

development of opti<strong>on</strong>al acti<strong>on</strong>s targeted to increase opportunities and to reduce threats to project<br />

activities. There are various strategies that can be implemented, including avoid, transfer, reduce, or<br />

accept threat risks and exploit, share, enhance, or accept opportunity risks. The risk c<strong>on</strong>trol process<br />

tracks identified risks, m<strong>on</strong>itors residual risks, identifies new risks and executes risk resp<strong>on</strong>se plans.<br />

Risk management outputs should be updated and communicated to relevant stakeholders throughout<br />

the project lifecycle.<br />

The initiating and probably the most important phase of risk management is risk identificati<strong>on</strong>. This<br />

phase involves the detecti<strong>on</strong> and classificati<strong>on</strong> of all known, and as much as possible also unknown,<br />

risks, thus producing the foundati<strong>on</strong> <strong>on</strong> which the whole risk management process can be established<br />

(Chapman, 2001). Therefore, the current study focuses <strong>on</strong> tools to enhance.<br />

2.2 History-based methods for risk identificati<strong>on</strong><br />

Organizati<strong>on</strong>al informati<strong>on</strong> and knowledge, which is a product of historical occurrences, provides a<br />

wide basis for analysis of future possible events. Historical informati<strong>on</strong> analysis techniques are derived<br />

either from investigati<strong>on</strong> of experiences that happened to employees and managers, or from the<br />

investigati<strong>on</strong> of documents that were produced c<strong>on</strong>tinuously by various stakeholders recording<br />

different activities. History-based methods include checklists, interviews, and documentati<strong>on</strong> reviews.<br />

A Checklist is an easy-to-use and effective aid which represents a list of possible risks, based <strong>on</strong><br />

historical informati<strong>on</strong> and accumulated organizati<strong>on</strong>al knowledge. A generic checklist may be<br />

generated from published research such as Caper J<strong>on</strong>es's software risks, Rex Black's quality risks, or<br />

Barry Boehm's top ten risk list (Ravindranath, 2007). Alternatively, the checklist can be provided by an<br />

established professi<strong>on</strong>al associati<strong>on</strong> such as the Software Engineering Institute (SEI, 2005), the<br />

Nati<strong>on</strong>al Institute of Standards and Technology (NIST, 2002), or other professi<strong>on</strong>al institutes in the<br />

industry. A specific checklist might be produced by an organizati<strong>on</strong>, thus capturing particular risks that<br />

previous projects within the organizati<strong>on</strong> have been exposed to (Hills<strong>on</strong>, 2004).<br />

Interviews are c<strong>on</strong>ducted in order to capture existing knowledge by talking to people. Project team<br />

members and managers, experienced project participants, project stakeholders and professi<strong>on</strong>al<br />

experts are interviewed to produce a comprehensive list of risks (Zsidisin, Panelli, and Upt<strong>on</strong>2000).<br />

Documentati<strong>on</strong> Reviews include a set of techniques by which organizati<strong>on</strong>al and project<br />

documentati<strong>on</strong> related to various types of activities is reviewed to analyze historical events for better<br />

future management. Typical documents for review are less<strong>on</strong>s learned files or debriefing registers,<br />

which record past experiences and can be used to enrich organizati<strong>on</strong>al knowledge (Kerzner, 1999;<br />

Williams, 2003).<br />

The current study is based <strong>on</strong> the c<strong>on</strong>cept that there is a synergetic linkage between the past and the<br />

future. In order to be better prepared for future experiences we need to understand past occurrences.<br />

The research methodology relies up<strong>on</strong> a history-based documentati<strong>on</strong>, which is analyzed and<br />

investigated in an ordered way to infer knowledgeable potential risks.<br />

3. The research methodology for analyzing less<strong>on</strong>s learned to identify<br />

potential risks<br />

The research methodology is based <strong>on</strong> two research techniques: c<strong>on</strong>tent analysis and cluster analysis.<br />

The first two steps include qualitative and quantitative c<strong>on</strong>tent analysis (Krippendorff, 2004; Neuendorf,<br />

2002). Qualitative c<strong>on</strong>tent analysis is a process of understanding, interpreting, analyzing and coding of<br />

documents. In the current study, this process was performed using the Atlas.ti software package <strong>on</strong> an<br />

especially developed codebook c<strong>on</strong>structed of items from the project management body of knowledge<br />

(PMI, 2008) with additi<strong>on</strong>al items related to business process flow (Jest<strong>on</strong> and Nelis, 2006). The<br />

qualitative analysis requires reading each document, understanding it in the c<strong>on</strong>text it was written, and<br />

categorizing by subjects or c<strong>on</strong>cepts. In the current study, the text units for analysis were the collected<br />

129


Vered Holzmann<br />

less<strong>on</strong>s-learned documents. The coding process was c<strong>on</strong>ducted using a specially-developed<br />

codebook that c<strong>on</strong>tained 200 codes derived from the current body of knowledge in project<br />

management. Quantitative c<strong>on</strong>tent analysis is based <strong>on</strong> the same text units, though requires statistical<br />

analysis. It c<strong>on</strong>tains word and phrases crunching, and descriptive frequency analyses assigned to<br />

investigate the occurrences of codes, groups of codes and words. The quantitative analysis results<br />

present numerical examinati<strong>on</strong>s of the interpreted text units and the related categorized codes, such as<br />

word count, code frequency analysis, and other categories frequency analyses. The output of these<br />

two analyses is a structured dataset of all the risks that were identified in previous less<strong>on</strong>s learned as<br />

gathered from an array of historical projects performed and managed by the organizati<strong>on</strong>.<br />

The following phase is clustering. Cluster analysis is performed in this research by a supervised<br />

classificati<strong>on</strong> of patterns derived from the qualitative and quantitative c<strong>on</strong>tent analyses (Jain, Murty and<br />

Flynn, 1999). The clustering techniques were investigated and the Ward’s method, using the Dice<br />

distance similarity measure, was found as the most appropriate <strong>on</strong>e for this purpose. Clustering is an<br />

advanced statistical method targeted to represent data, based <strong>on</strong> measures of proximity between<br />

elements, expressed by maximum distances between groups and minimal distances within each<br />

group. Joining or tree clustering algorithms, which were used in this study, are designed to join objects<br />

into sequentially larger groups based <strong>on</strong> some measure of similarity or distance. The distances<br />

represent a set of rules that functi<strong>on</strong> as criteria for grouping or separating objects, and they can be<br />

composed of <strong>on</strong>e or more sets of rules or c<strong>on</strong>diti<strong>on</strong>s. The output of this analysis was a hierarchical<br />

RBS (Risk Breakdown Structure) that represents the project and organizati<strong>on</strong>al risks, arranged in<br />

hierarchical order where every set of risk events are grouped under a comm<strong>on</strong> risk factor.<br />

An RBS (Risk Breakdown Structure) is defined by David Hills<strong>on</strong> (2002) as: “A source-oriented grouping<br />

of risks that organizes and defines the total risk exposure of the project or business. Each descending<br />

level represents an increasingly detailed definiti<strong>on</strong> of sources of risk”. The RBS is a hierarchical<br />

structure of risk factors and events.<br />

The following chart represents the research methodology steps.<br />

Figure 2: The research methodology<br />

4. Case study: An IT company<br />

The research methodology was implemented in a High-Tech company in Israel. The company is an IT<br />

service provider, which is specializing in end-to-end soluti<strong>on</strong>s in all market sectors. The company is a<br />

top Israeli IT provider. It is a public company trading <strong>on</strong> the Tel Aviv Stock Exchange and employs<br />

over 2,500 experts. Its services include outsourcing, software development, ERP system applicati<strong>on</strong>s,<br />

payroll and human resource soluti<strong>on</strong>s, technical support and helpdesk, etc. The company provided an<br />

access to its less<strong>on</strong>s learned documents which were accumulated during 12 m<strong>on</strong>ths.<br />

4.1 Qualitative c<strong>on</strong>tent analysis<br />

In order to dem<strong>on</strong>strate the qualitative c<strong>on</strong>tent analysis process, a representative less<strong>on</strong> learned<br />

document, which is <strong>on</strong>e of the 40 less<strong>on</strong>s learned documents of the IT Company that was analyzed in<br />

the research, is presented.<br />

Introducti<strong>on</strong>: The project was designed to analyze and implement the central system software<br />

upgrading procedure of a very busy mainframe that supplied <strong>on</strong>line services to a large array of users.<br />

130


Vered Holzmann<br />

The project was postp<strong>on</strong>ed several times due to customer reluctance to permit service technicians to<br />

perform any alterati<strong>on</strong> <strong>on</strong> the brittle state of the original settings of its system software. The project<br />

was very sensitive since no updates to the original installati<strong>on</strong> were ever performed. Thus,<br />

implementati<strong>on</strong> required an extremely well designed plan, excepti<strong>on</strong>ally skilled professi<strong>on</strong>als and a<br />

very thorough testing program. In additi<strong>on</strong>, the total project set <strong>on</strong> the critical path, thus there was a<br />

need for a stringent risk mitigati<strong>on</strong> plan based <strong>on</strong> a wide identificati<strong>on</strong> of risk events followed by a<br />

progressive occurrence forecast.<br />

Story: A qualified team of technicians simulated, as close as possible, the actual resident envir<strong>on</strong>ment<br />

at the customer’s labs. A complete array of tests was performed. A detailed log of every occurrence<br />

was maintained. Problems of any nature were discussed first within the local divisi<strong>on</strong> and further<br />

clarified with the vendor’s global technical department. Finally the pers<strong>on</strong>al matrix of resp<strong>on</strong>sibility<br />

was assign and tested. The preparati<strong>on</strong>s for the event were perfect, and then everything that could go<br />

wr<strong>on</strong>g actually did go wr<strong>on</strong>g. The executi<strong>on</strong> was postp<strong>on</strong>ed several times, <strong>on</strong>ly hours before due<br />

date, after everything was already put in place. Hence various pers<strong>on</strong>nel shifts were necessary, which<br />

demoralized the team and seriously shook the planned subordinati<strong>on</strong> structure. At last the upgrading<br />

started. So<strong>on</strong> uncharted bugs appeared and hampered system performance. After c<strong>on</strong>sultati<strong>on</strong> the<br />

team manager decided they could fix the modules, and proceeded with the upgrade. However<br />

immediately after passing the point of no return, namely, the point where the old system could no<br />

l<strong>on</strong>ger be reinstated it became evident that the new upgraded <strong>on</strong>e did not work properly. The<br />

customer was informed, and then everybody involved became panic-stricken. The vendor c<strong>on</strong>tacted<br />

all his global users and asked them to participate in an internati<strong>on</strong>al temporary c<strong>on</strong>sortium to solve<br />

the current problem. At the eleventh hour of the project’s maximum expected durati<strong>on</strong>, following<br />

intense global communicati<strong>on</strong>s, a software engineer, located in India, came up with the answers and<br />

the upgrade went through.<br />

Coding: The following codes were assigned.<br />

Table 1: A sample of a code assignment to a less<strong>on</strong>s learned document<br />

4.2 Quantitative c<strong>on</strong>tent analysis<br />

Code Name<br />

1090 define process procedures<br />

1110 define stakeholders involvement<br />

1120 define stakeholders expectati<strong>on</strong>s<br />

1150 define critical success factors<br />

1170 involve customer in process method definiti<strong>on</strong><br />

1180 involve customer in deliverables definiti<strong>on</strong><br />

1190 involve customer in success criteria definiti<strong>on</strong><br />

1200 involve customer in process limitati<strong>on</strong>s definiti<strong>on</strong><br />

1230 define product attributes<br />

1240 define delivery method<br />

1290 define work c<strong>on</strong>straints<br />

1330 define required documentati<strong>on</strong><br />

1360 define wp (work package) prerequisites<br />

1430 define wp c<strong>on</strong>straints<br />

2070 define quality requirements<br />

2110 define QA procedures<br />

2170 plan tests<br />

2330 identify risk factors<br />

2340 identify risk events<br />

2350 manage risk assessment<br />

2360 assess risk probability<br />

2370 assess risk impact<br />

2850 attenti<strong>on</strong> to c<strong>on</strong>straint indicators by HR<br />

2860 attenti<strong>on</strong> to guidance tools by HR<br />

The basic quantitative analysis is a code frequency analysis. For the IT service provider this analysis<br />

was based <strong>on</strong> the assignment of 117 research codes (out of the 200 that appear in the codebook) to<br />

the 40 less<strong>on</strong>s learned documents that were generated during the year 2007. The frequency analysis<br />

of the codes assigned to the less<strong>on</strong>s learned of the IT company reveals that the most frequent items,<br />

which c<strong>on</strong>stitute about 19% of the total code assignments, are: define work package (wp)<br />

131


Vered Holzmann<br />

resp<strong>on</strong>sibilities, define informati<strong>on</strong> transfer resp<strong>on</strong>sibilities, define work instructi<strong>on</strong>s and c<strong>on</strong>straints,<br />

define informati<strong>on</strong> transfer requirements, distribute informati<strong>on</strong> to team members, c<strong>on</strong>trol IT<br />

performance and optimizati<strong>on</strong>. This list of items can be summarized into topics: resp<strong>on</strong>sibilities,<br />

communicati<strong>on</strong> and resource planning and c<strong>on</strong>trolling.<br />

Analysis of the codes assigned less<strong>on</strong>s learned documents, as related to process group yields the<br />

following results.<br />

Figure 3: Process group frequency chart<br />

An analysis of the distributi<strong>on</strong> of the codes related to planning identifies the codes of define wp<br />

resp<strong>on</strong>sibilities and define informati<strong>on</strong> transfer resp<strong>on</strong>sibilities as the most frequent <strong>on</strong>es. The<br />

sec<strong>on</strong>dary frequent codes are: define work instructi<strong>on</strong>s, define work c<strong>on</strong>straints, and define<br />

informati<strong>on</strong> transfer requirements. Hence, the planning process, especially related to the topics of<br />

work planning and communicati<strong>on</strong>s planning, is obviously problematic in the IT organizati<strong>on</strong>. Am<strong>on</strong>g<br />

the m<strong>on</strong>itoring and c<strong>on</strong>trolling process group the most frequent codes were c<strong>on</strong>trol of IT performance<br />

and optimizati<strong>on</strong>. Within the executing process group the most frequent code was distribute<br />

informati<strong>on</strong> to team members. The most frequent codes within the initiating process group were<br />

related to customer involvement and deliverable definiti<strong>on</strong>.<br />

Figure 4: Dendogram representati<strong>on</strong> of hierarchical risk clustering<br />

The visualized representati<strong>on</strong>s of the cluster analysis results are displayed in this dendogram chart. It<br />

depicts the RBS (Risk Breakdown Structure) of the IT organizati<strong>on</strong> as derived from the c<strong>on</strong>tent<br />

analysis of its less<strong>on</strong>s learned documents. The RBS hierarchical representati<strong>on</strong> suggests an<br />

interesting classificati<strong>on</strong> of diversified risk items. The first level differentiates between issues of<br />

customer involvement and communicati<strong>on</strong>s <strong>on</strong> <strong>on</strong>e hand, and product descripti<strong>on</strong> and the work<br />

required to produce it <strong>on</strong> the other hand. Each of these two major branches is further divided. The first<br />

branch refers to issues of infrastructure and management support; the sec<strong>on</strong>d branch focuses <strong>on</strong><br />

quality procedures and c<strong>on</strong>trol analysis processes; the next branch refers to identificati<strong>on</strong> of<br />

132


Vered Holzmann<br />

c<strong>on</strong>straints; the following branch is involved with change management and risk management; the fifth<br />

branch c<strong>on</strong>centrates <strong>on</strong> specific resource planning items; the next <strong>on</strong>e relates to accurate definiti<strong>on</strong>s<br />

of the required work during the planning phase; and the following branch is directed towards a<br />

definiti<strong>on</strong> of the required resources; the two final branches are: issues related to human resource<br />

management; and, issues of informati<strong>on</strong> exchange.<br />

5. C<strong>on</strong>clusi<strong>on</strong>s<br />

The current study presents a methodology that integrates qualitative and quantitative methods to<br />

enable the identificati<strong>on</strong> of risk factors in high-tech companies. The methodology is based <strong>on</strong> the<br />

analysis of existing organizati<strong>on</strong>al informati<strong>on</strong> while transforming it into valuable managerial<br />

knowledge. The c<strong>on</strong>cept that associates risk management and knowledge management was<br />

articulated by Neef (2005) who argued that risk management is knowledge management. This<br />

research encompasses both risk management and knowledge management, which is a c<strong>on</strong>cept that<br />

is also supported by other studies (e.g., Kerzner, 1999; Williams, 2003; Perrott, 2007), into <strong>on</strong>e<br />

homogeneous method that starts with existing informati<strong>on</strong> and ends with knowledge that emerges as<br />

a risk management plan.<br />

The integrati<strong>on</strong> of qualitative and quantitative methods into <strong>on</strong>e methodology yielded an in-depth<br />

understanding of the project and organizati<strong>on</strong>al risk factors and risk events that an organizati<strong>on</strong> might<br />

encounter. This understanding is of great importance especially in new product development projects<br />

(Grubisic, Gidel and Ogliari, 2011) due to the high level of uncertainty involved in this type of projects.<br />

It is interesting to find out that even in high technology projects the major influence of the identified<br />

risks was nevertheless related to the planning phase of communicati<strong>on</strong>s, which manifested itself in a<br />

later phase as problems with human resource performance.<br />

References<br />

Chapman, C. B. (2001) “The C<strong>on</strong>trolling Influences <strong>on</strong> Effective Risk Identificati<strong>on</strong> an Assessment for<br />

C<strong>on</strong>structi<strong>on</strong> Design <strong>Management</strong>”, Internati<strong>on</strong>al Journal of Project <strong>Management</strong>, Vol.19, No.3, pp.147-160.<br />

Cohen, M.W. and Palmer, G.R. (2004) "Project Risk Identificati<strong>on</strong> and <strong>Management</strong>", AACE Internati<strong>on</strong>al<br />

Transacti<strong>on</strong>s, IN11-IN15.<br />

Cook, P. (2005) "Formalized Risk <strong>Management</strong>: Vital Tool for Project and Business Success", Cost Engineering,<br />

Vol. 47 No. 8, pp. 12-13.<br />

COSO (Committee of Sp<strong>on</strong>soring Organizati<strong>on</strong>s of the Treadway Commissi<strong>on</strong>) (2004) Enterprise Risk<br />

<strong>Management</strong> – Integrated Framework: Executive Summary, AICPA, New York, NY.<br />

Garrett, G. A. (2005) “Managing Opportunity & Risk in a Complex Project Envir<strong>on</strong>ment”, C<strong>on</strong>tract <strong>Management</strong>,<br />

Vol.54, No.4, pp.8-20.<br />

Grubisic, V.V.F., Gidel, T. and Ogliari, A. (2011) " Recommendati<strong>on</strong>s for risk identificati<strong>on</strong> method selecti<strong>on</strong><br />

according to product design and project management maturity, product innovati<strong>on</strong> degree and project<br />

team", ICED 11 - 18th Internati<strong>on</strong>al <str<strong>on</strong>g>C<strong>on</strong>ference</str<strong>on</strong>g> <strong>on</strong> Engineering Design - Impacting Society Through<br />

Engineering Design Vol. 3, 2011, pp. 187-198.<br />

Higuera, R.P. and Haimes, Y.Y. (1996) Software Risk <strong>Management</strong>, Technical Report, Software Engineering<br />

Institute, Carnegie Mell<strong>on</strong> University, Pittsburgh, PA.<br />

Hill, G. M. (2008) The Complete Project <strong>Management</strong> Office Handbook, Taylor & Francis Group, Boca Rat<strong>on</strong>, FL.<br />

Hills<strong>on</strong>, D. (2004) Effective Opportunity <strong>Management</strong> for Projects: Exploiting Positive Risk, CRC Press, Taylor &<br />

Francis Group, Boca Rat<strong>on</strong>, FL.<br />

Hills<strong>on</strong>, D. (2002) “Using the Risk Breakdown Structure (RBS) to Understand Risks”, Proceedings of the 33rd<br />

Annual Project <strong>Management</strong> Institute Seminars & Symposium (PMI 2002), San Ant<strong>on</strong>io, USA, 7th-8th<br />

October, Philadelphia, PMI.<br />

Jain, A.K., Murty, M.N. and Flynn, P.J. (1999) “Data Clustering: A Review”, ACM Computing Surveys, Vol.31,<br />

No.3, pp.264-323.<br />

Jest<strong>on</strong>, J.and Nelis, J. (2006) Business Process <strong>Management</strong>: Practical Guidelines to Successful<br />

Implementati<strong>on</strong>s, Butterworth-Heinemann, Jordan Hill, Oxford.<br />

Kerzner, H. (2005) Using the Project <strong>Management</strong> Maturity Model: Strategic Planning for Project <strong>Management</strong><br />

(2nd Ed.) John Wiley & S<strong>on</strong>s, Hoboken, NJ.<br />

Kerzner Harold (1999) Applied Project <strong>Management</strong>: Best Practices <strong>on</strong> Implementati<strong>on</strong>, John Wiley & S<strong>on</strong>s, Inc.,<br />

New York, NY.<br />

Krippendorff, K. (2004) C<strong>on</strong>tent Analysis: An Introducti<strong>on</strong> to Its Methodology, 2nd ed., Sage Publicati<strong>on</strong>s,<br />

Newbury Park, CA.<br />

Lipshitz, R., Popper, M., and Friedman, V.J. (2008) "A Multifacet Model of Organizati<strong>on</strong>al Learning" Journal of<br />

Applied Behavioral Science, Vol. 38, No.1, pp. 78-98.<br />

Marxt, C. and Link, P. (2002) "Success factors for Cooperative ventures in Innovati<strong>on</strong> Producti<strong>on</strong> Systems",<br />

Internati<strong>on</strong>al Journal of Producti<strong>on</strong> Ec<strong>on</strong>omics, Vol. 77, Iss. 3, pp. 219-229.<br />

Nalewaik, A. (2005) "Risk <strong>Management</strong> for Pharmaceutical Project Schedules", AACE Internati<strong>on</strong>al Transacti<strong>on</strong>s,<br />

Risk07, pp. 71-75.<br />

133


Vered Holzmann<br />

Neef, D. (2005) “Managing Corporate Risk through Better Knowledge <strong>Management</strong>”, The Learning Organizati<strong>on</strong>,<br />

Vol.12, No.2, pp.112-124.<br />

NIST (Nati<strong>on</strong>al Institute of Standards and Technology) (2002) Risk <strong>Management</strong> Guide for Informati<strong>on</strong> and<br />

Technology Systems, [SP 800-30] Computer Security Divisi<strong>on</strong>, IT Laboratory, Gaithersburg, MD.<br />

Neuendorf, K. A. (2002) The C<strong>on</strong>tent Analysis Guidebook, Sage Publicati<strong>on</strong>s, Thousand Oaks, CA.<br />

Olss<strong>on</strong> R. (2007) “In Search of Opportunity <strong>Management</strong>: Is the Risk <strong>Management</strong> Process Enough?”<br />

Internati<strong>on</strong>al Journal of Project <strong>Management</strong>, Vol.25, No.8, pp.745-752.<br />

Perrott, B.E. (2007) “A Strategic Risk Approach to Knowledge <strong>Management</strong>”, Business Horiz<strong>on</strong>, Vol.50, pp.523-<br />

533.<br />

PMI (Project <strong>Management</strong> Institute) Standards Committee (2008) A Guide to the Project <strong>Management</strong> Body of<br />

Knowledge (PMBOK® Guide) 4th ed., Project <strong>Management</strong> Institute, Newtown Square, PA.<br />

Ravindranath, C. P. (2007) Applied Software Risk <strong>Management</strong>, Taylor & Francis Group, Boca Rat<strong>on</strong>, FL.<br />

Raz, T., Shenhar, A.J. and Dvir, D. (2002) "Risk <strong>Management</strong>, Project Success, and Technological Uncertainty",<br />

R&D <strong>Management</strong>, Vol. 32, No. 2, pp. 101-110.<br />

SEI (Software Engineering Institute) A Tax<strong>on</strong>omy of Operati<strong>on</strong>al Risks, 15213-3890 [CMU/SEI-2005-TN-036],<br />

Carnegie Mell<strong>on</strong> University, Pittsburgh, PA.<br />

Ward, S. C. and Chapman, C. B. (2003) "Transforming Project Risk <strong>Management</strong> into Project Uncertainty<br />

<strong>Management</strong>", Internati<strong>on</strong>al Journal of Project <strong>Management</strong>, Vol.21, No.2, pp.97-105.<br />

Williams, T. (2003) “Learning from Projects”, Journal of the Operati<strong>on</strong>al Research Society, Vol.54, No.5, pp.443-<br />

451.<br />

Zsidisin, G.A., Panelli, A. and Upt<strong>on</strong>, R. (2000) “Purchasing Organizati<strong>on</strong> Involvement in Risk Assessment,<br />

C<strong>on</strong>tingency Plans, and Risk <strong>Management</strong>: An Exploratory Study”, Supply Chain <strong>Management</strong>, Vol.5, No.4,<br />

pp.187-197.<br />

134


Evaluating Determinants for ERP use and Value in Scandinavia:<br />

Exploring Differences Between Danish and Swedish<br />

SMEs<br />

Björn Johanss<strong>on</strong> 1 , Pedro Ruivo 2 , Tiago Oliveira 2 and Miguel Neto 2<br />

1<br />

Department of Informatics, Lund University, Sweden<br />

2<br />

ISEGI, Universidade Nova de Lisboa, Portugal<br />

bjorn.johanss<strong>on</strong>@ics.lu.se<br />

pruivo@isegi.unl.pt<br />

toliveira@isegi.unl.pt<br />

mneto@isegi.unl.pt<br />

Abstract: In the paper we present a research model for evaluating determinants of ERP value in small and medium-size<br />

enterprises (SMEs). The model is grounded in the diffusi<strong>on</strong> of innovati<strong>on</strong> (DOI) model and resourcebased<br />

view of the firm (RBV) theory. The research model links six DOI determinants to explain ‘ERP use’ linked<br />

with two additi<strong>on</strong>al determinants to explain ‘ERP value’, <strong>on</strong> which nine hypotheses are postulated. The hypotheses<br />

were tested through structural equati<strong>on</strong> modelling <strong>on</strong> a dataset from a web survey of 325 SMEs in Denmark<br />

(107) and Sweden (218). Through an empirical work we validate the theoretical arguments and provide insight<br />

into how SMEs use and value ERP, especially how perceived ERP use and perceived ERP value in Scandinavian<br />

SMEs can be explained. To our knowledge this is the first empirical research study <strong>on</strong> Scandinavian SMEs,<br />

thus adding a cross-country dimensi<strong>on</strong> to the innovati<strong>on</strong> diffusi<strong>on</strong> literature. Unlike the typical focus <strong>on</strong> ERP<br />

adopti<strong>on</strong> in large firms found in the literature, this study focuses <strong>on</strong> post-adopti<strong>on</strong> of ERP in SMEs. The main finding<br />

is that Danish and Swedish SMEs show different results despite the fact that they seem to be so similar. Our<br />

study reveals that while transacti<strong>on</strong>al efficiency, best-practices, and competitive pressure are important determinants<br />

of ‘ERP use’ in both Swedish and Danish SMEs, complexity is significant <strong>on</strong>ly am<strong>on</strong>g Danish firms. Compatibility<br />

has c<strong>on</strong>trary effects, i.e., it is an inhibitor for Danish SMEs and a facilitator for Swedish SMEs to explain<br />

ERP use. Furthermore, while for Danish SMEs ‘ERP value’ is explained mainly by collaborati<strong>on</strong>, for Swedish<br />

SMEs ‘ERP value’ is explained mainly by analytics. The facts that the research presents results focusing <strong>on</strong><br />

SMEs makes it especially valuable since this is an under researched area, and that the research has 325 resp<strong>on</strong>dents<br />

also makes it important in exploring the differences and similarities between countries, adding an internati<strong>on</strong>al<br />

dimensi<strong>on</strong>.<br />

Keywords: ERP, SMEs, diffusi<strong>on</strong> of innovati<strong>on</strong>, resource-based view, use, value, post-adopti<strong>on</strong><br />

1. Introducti<strong>on</strong><br />

Some enterprise systems (ES), such as enterprise resource planning systems (ERPs) are more or<br />

less a de facto standard am<strong>on</strong>gst large organizati<strong>on</strong>s. Lately, small and medium-sized enterprises<br />

(SMEs) have also showed a great interest in adopting ERPs. However, in the c<strong>on</strong>text of SMEs an interesting<br />

questi<strong>on</strong> is how decisi<strong>on</strong>-makers in SMEs perceive the value of the adopted system. The<br />

questi<strong>on</strong>s addressed in this paper are: how can perceived ERP use and perceived ERP value in<br />

SMEs be explained? From the results of those questi<strong>on</strong>s we then explore if there are differences between<br />

Danish and Swedish SMEs. Through this comparis<strong>on</strong> we seek to increase the knowledge <strong>on</strong><br />

determinants for what is seen as influencing gained value from a decisi<strong>on</strong> <strong>on</strong> adopti<strong>on</strong> of ERPs in<br />

SMEs. This is of interest to stakeholders in the ERP value-chain. The vendors and providers of ERPs<br />

will gain a better understanding of what user organizati<strong>on</strong>s experience about gained value in the postadopti<strong>on</strong><br />

phase of an ERP. The user organizati<strong>on</strong>s and also organizati<strong>on</strong>s that have not yet decided<br />

about adopti<strong>on</strong> of ERP will gain knowledge <strong>on</strong> what they can expect as a result of adopting an ERP<br />

system.<br />

We present a research model for evaluating determinants of ERP value in SMEs. The model is<br />

grounded in the diffusi<strong>on</strong> of innovati<strong>on</strong> (DOI) model and resource-based view of the firm (RBV) theory.<br />

The research model links six DOI determinants to explain ‘ERP use’ linked with two additi<strong>on</strong>al determinants<br />

to explain ‘ERP value’, <strong>on</strong> which nine hypotheses are postulated Testing of the hypotheses<br />

was c<strong>on</strong>ducted through structural equati<strong>on</strong> modelling <strong>on</strong> a dataset from a web survey of 325 SMEs in<br />

Denmark and Sweden.<br />

The remainder of the paper is structured as follows. The next secti<strong>on</strong> presents a literature review giving<br />

the theoretical perspectives as well as the research design and hypotheses. The characteristics of<br />

135


Björn Johanss<strong>on</strong> et al.<br />

the survey data and the results are presented in Secti<strong>on</strong> 3. Secti<strong>on</strong> 4 then analyse the hypotheses.<br />

The final secti<strong>on</strong> then presents c<strong>on</strong>clusi<strong>on</strong>s and some ideas for future research.<br />

2. Theoretical perspectives and hypotheses<br />

According to Nah, Tan and Teh (2004) ERP use means ERP utilizati<strong>on</strong>, which refers to the experience<br />

of managing the operati<strong>on</strong> of the system software throughout the system’s post-implementati<strong>on</strong><br />

stages. In line with literature, we c<strong>on</strong>sider ERP to be a type of innovati<strong>on</strong> that is implanted in a firm’s<br />

core business processes in order to leverage performance (Rajagopal, 2002; Zhu and Kraemer,<br />

2005). Not <strong>on</strong>ly does it extend basic business and streamline integrati<strong>on</strong> with suppliers and customers,<br />

it also directs system usage to the firm’s performance. Rogers' (1995) Diffusi<strong>on</strong> of innovati<strong>on</strong><br />

(DOI) model aims to explain and predict if and how an innovati<strong>on</strong> is used within a social system, with<br />

regard to performance at the firm level. Research c<strong>on</strong>ducted by Bradford and Florin (2003) verifies<br />

DOI determinants regarding successful ERP usage. C<strong>on</strong>sidering their findings, we believe that DOI<br />

has the potential to provide a favourable framework for explaining ERP use.<br />

As informati<strong>on</strong> technology (IT) value relies <strong>on</strong> how firms strategically exploit it, firms’ performance in a<br />

competitive envir<strong>on</strong>ment is a subject that draws much attenti<strong>on</strong> and attempts to build explanatory<br />

theories. One of the most recognized is the resource-based view (RBV) theory, which states that firmspecific<br />

resources determine the firm’s performance. It is linked to the competitive advantage approach<br />

to strategic management and can explain sustained advantages (Hedman and Kalling, 2003).<br />

In the IS literature, the RBV has been used to analyse IT capabilities as a resource and to explain IT<br />

business value. That is, IT business value depends <strong>on</strong> the extent to which IT is used in the key activities<br />

of the firm. The greater the use, the more likely the firm is to develop unique capabilities from its<br />

IT business applicati<strong>on</strong>s (Antero and Riis, 2011; Bharadwaj, 2000; Zhu and Kraemer, 2005). Hedman<br />

and Kalling (2003) and Fosser et al. (2008) used RBV to extend Mata et al.’s (1995) framework for<br />

organizati<strong>on</strong>al and business resources and c<strong>on</strong>cluded that ERP systems are IT resources that can<br />

lead to sustained, competitive advantages. With this in mind, we believe that RBV has the potential to<br />

provide a favourable framework for explaining ERP value.<br />

For these reas<strong>on</strong>s, we next postulate six hypotheses to explain ERP use (Hypothesis 1 – Hypothesis<br />

6) based <strong>on</strong> DOI literature, and three hypotheses based <strong>on</strong> RBV theory to explain ERP value (Hypothesis<br />

7 – Hypothesis 9).<br />

Hypothesis 1: ERP systems with high compatibility positively influence ERP use.<br />

Compatibility is measured by the degree to which the ERP system matches IT features, such as compatibility<br />

with hardware and other software. Bradford and Florin (2003) and Elbertsen et al. (2006)<br />

c<strong>on</strong>cluded that the degree of compatibility of ERP will have a positive relati<strong>on</strong>ship with system adopti<strong>on</strong><br />

and use.<br />

Hypothesis 2: ERP systems with high complexity negatively influence ERP use.<br />

Cooper and Zmud's (1990) research indicates that system usage familiarity enhances job performance.<br />

Studies c<strong>on</strong>ducted by Kositanurit et al. (2006) and Chang et al. (2011) c<strong>on</strong>clude that ERP complexity<br />

is a major factor affecting user performance.<br />

Hypothesis 3: ERP systems with high transacti<strong>on</strong>al efficiency positively influence ERP use.<br />

Bendoly and Kaefer (2004) assessed transacti<strong>on</strong>al efficiency and found that its communicati<strong>on</strong> over<br />

the ERP improves the firm’s overall performance. Rajagopal (2002) found that transacti<strong>on</strong>al efficiency<br />

has a direct influence <strong>on</strong> ERP use. Business process benefits of ERP investment include transacti<strong>on</strong>al<br />

efficiency, where reliability effectiveness <strong>on</strong> the applicati<strong>on</strong> improves user c<strong>on</strong>fidence. Al<strong>on</strong>g the<br />

same lines, Gattiker and Goodhue (2005) found that efficiency greatly benefits ERP use.<br />

Hypothesis 4: ERP systems implemented best-practices positively influence ERP use.<br />

From the perspective of business process reengineering, there are two main opti<strong>on</strong>s in implementing<br />

ERP systems: modify/customize the system to suit the firm’s requirements, or implementing the system<br />

with minimum deviati<strong>on</strong> from the standard settings - adopting best practice - (Davenport, 1998).<br />

According to Chou and Chang (2008) and Maguire et al. (2010) the reas<strong>on</strong> for adopting ‘best-<br />

136


Björn Johanss<strong>on</strong> et al.<br />

practices’ is the belief that ERP design does things in the right way. In line with Wenrich and Ahmad<br />

(2009), firms that implement industry best-practices dramatically reduce risk and time c<strong>on</strong>suming project<br />

tasks such as c<strong>on</strong>figurati<strong>on</strong>, documentati<strong>on</strong>, testing, and training. Thus, we postulate that bestpractices<br />

will positively influence ERP use.<br />

Hypothesis 5: User training of ERP systems will positively influence ERP use.<br />

Several researchers, including O’Leary (2000), Bradford and Florin (2003), and Maguire et al. (2010)<br />

state that <strong>on</strong>e of the main determinants for the successful use of ERPs is training of users. They state<br />

that preparedness of users to carry out a planned sequence of acti<strong>on</strong>s without upstream errors has a<br />

positive impact <strong>on</strong> business. Providing knowledge and skills to employees <strong>on</strong> how to use the system<br />

improves familiarity and boosts its usage.<br />

Hypothesis 6: Competitive pressure positively influences ERP use.<br />

Competitive pressure has l<strong>on</strong>g been recognized in the innovati<strong>on</strong> diffusi<strong>on</strong> literature as an important<br />

driver of technology diffusi<strong>on</strong> (Bradford and Florin, 2003; Oliveira and Martins, 2010b; Zhu and<br />

Kraemer, 2005). These studies have shown that innovati<strong>on</strong> diffusi<strong>on</strong> is accelerated by the competitive<br />

pressure in the envir<strong>on</strong>ment. Thus, we postulate that competitive pressure plays an important role in<br />

pushing firms toward using ERP systems.<br />

Hypothesis 7: ERP use positively influences ERP value.<br />

Shahin and Ainin (2011) found that user fit <strong>on</strong> ERP is important to explain the ERP usage, and a successful<br />

adaptati<strong>on</strong> with firms’ processes and data flow from other systems makes ERP worthwhile.<br />

With ERP systems firms can form a specific resource that guides both internal and external collaborati<strong>on</strong><br />

and provides the repository to perform business analyses. As a result, it is <strong>on</strong>ly when firms actually<br />

use ERP systems to c<strong>on</strong>duct business that it can have an impact <strong>on</strong> firm performance (Devaraj<br />

and Kohli, 2003; Zhu and Kraemer, 2005).<br />

Hypothesis 8: Collaborati<strong>on</strong> by ERP systems positively influences ERP value.<br />

Calisir and Calisir (2004), Gattiker and Goodhue (2005), and Ruivo et al. (2012) support the c<strong>on</strong>clusi<strong>on</strong><br />

that ERP systems help users to collaborate; up, down, and across their department, company,<br />

and industry ecosystem, increasing their productivity and the health of their firms and business partners.<br />

ERP allows both humans and applicati<strong>on</strong>s to collaborate, from meeting service-level to supporting<br />

firm’s performance. ERP provides users with a structured communicati<strong>on</strong> channel with the right informati<strong>on</strong><br />

at the right time, resulting in increased efficiency and effectiveness. We believe that<br />

partnering with system and cross-group collaborati<strong>on</strong> amplifies ERP value.<br />

Hypothesis 9: Analytics exploited from ERP systems positively influence ERP value.<br />

Davenport and Harris (2007) stated that firms generally use business analytics to gain competiveness.<br />

The comm<strong>on</strong> data model and visibility across functi<strong>on</strong>al departments allows firms’ metrics/KPIs<br />

to be unified and c<strong>on</strong>sistent. Although ERP systems are essentially transacti<strong>on</strong>-focused <strong>on</strong> internal<br />

data, those firms that exploit ERP analytics capabilities can easily and quickly use data for managerial<br />

decisi<strong>on</strong> making and realize an advantage in their pursuit of sustainable performance (Chiang, 2009;<br />

Ruivo and Neto, 2010). We therefore suggest that analytics provides users with unique business insight<br />

informati<strong>on</strong>, and thereby positively influences ERP value.<br />

3. Research methodology and data analysis<br />

A web-survey was used for data collecti<strong>on</strong> and each item-questi<strong>on</strong> was reviewed for c<strong>on</strong>tent validity<br />

by ERP experts. The initial questi<strong>on</strong>naires were pilot tested <strong>on</strong> 10 firms and some items were revised<br />

for clarity with assistance from the Internati<strong>on</strong>al Data Corporati<strong>on</strong>. To ensure the generalizati<strong>on</strong> of the<br />

survey results, the sampling was stratified by country (Denmark and Sweden), by firm size (fewer than<br />

250 employees), and by industry (finance, distributi<strong>on</strong>, manufacturing, and professi<strong>on</strong>al-services).<br />

Questi<strong>on</strong>naires were translated into the two languages and sent in September and October 2011. In<br />

total, 600 (200 Danish and 400 Swedish) firms received the email survey, and 325 (107 Danish and<br />

218 Swedish) valid resp<strong>on</strong>ses were returned. Table 1 shows characteristics of the sample, regarding<br />

number of years using ERP, industry type, and positi<strong>on</strong> of resp<strong>on</strong>dent. In general it can be stated that<br />

137


Björn Johanss<strong>on</strong> et al.<br />

there are more similarities between Denmark and Sweden than there are differences in the sample.<br />

The biggest difference in the sample is the number of years using ERP, where there is a higher frequency<br />

of SMEs that have used ERPs for more than ten years in Sweden.<br />

Table1: Characteristics of the samples<br />

Sweden (N=218) Denmark<br />

(N=107)<br />

# years using ERP<br />

10 42.2% 27.1%<br />

Industry type<br />

100% 100%<br />

Distributi<strong>on</strong> 27.5% 27.1%<br />

Finance 18.3% 21.5%<br />

Manufacturing 33.1% 23.4%<br />

Services 21.1% 28.0%<br />

Resp<strong>on</strong>dent type<br />

100% 100%<br />

CEO, owner 30.2% 22.4%<br />

Finance manager 18.8% 24.3%<br />

IT/IS manager 10.2% 14.9%<br />

Manufacturing manager 12.4% 8.5%<br />

Sales manager 28.4% 29.9%<br />

100% 100%<br />

The survey instrument was based <strong>on</strong> well-established scales as adapted to the c<strong>on</strong>text of ERP (see<br />

Appendix A). We performed the Kolmogorov-Smirnov test and c<strong>on</strong>firmed that n<strong>on</strong>e of the items<br />

measured are distributed normally (p


Björn Johanss<strong>on</strong> et al.<br />

are found to be important inhibitors for Danish SMEs firms when compatibility is a facilitator for Swedish<br />

SMEs. Regarding complexity, it is also an inhibitor for Swedish SMEs even if it is not seen as such<br />

a “str<strong>on</strong>g” inhibitor for Danish SMEs. Regarding ‘ERP value’; ‘ERP use’ is not a statistically significant<br />

difference (p>0.10) between countries, which means that ERP use is understood as being equally important<br />

for both countries, SMEs. Collaborati<strong>on</strong> is seen as more important for ERP value am<strong>on</strong>g Danish<br />

SMEs while analytics is a more important factor for Swedish SMEs when it comes to perceived<br />

ERP value.<br />

Figure 1: Research model, path models of Sweden and Denmark SMEs<br />

Table 2: Results of pooled error term t-tests<br />

Sweden Denmark Comparis<strong>on</strong><br />

(Sweden-Denmark)<br />

Path co- SE from Path co- SE from t-Stat. p (2eff.<br />

bootstrap eff. bootstrap<br />

tailed)<br />

Compatibility 0.152 0.035 -0.374 0.025 12.102 0.000<br />

Complexity -0.010 0.026 -0.109 0.034 2.350 0.019<br />

Efficiency<br />

Best-Practices<br />

ERP Use<br />

0.162<br />

0.173<br />

0.031<br />

0.032<br />

0.182<br />

0.370<br />

0.020<br />

0.033<br />

-0.559<br />

-4.262<br />

0.576<br />

0.000<br />

Training 0.032 0.040 0.000 0.025 0.686 0.493<br />

Competitive<br />

0.370 0.023 0.152 0.025 6.459 0.000<br />

ERP Use 0.024 0.023 -0.014 0.036 0.902 0.367<br />

Collaborati<strong>on</strong> ERP Value 0.343 0.023 0.505 0.028 -4.509 0.000<br />

Analytics<br />

0.361 0.032 0.215 0.031 3.275 0.001<br />

4. Discussi<strong>on</strong><br />

The purpose of this research was to identify determinants that explain ERP use and ERP value, and<br />

the magnitude variati<strong>on</strong> across Swedish and Danish SMEs. Empirical results support our theoretical<br />

139


Björn Johanss<strong>on</strong> et al.<br />

model, and all hypotheses have been tested. <strong>Academic</strong> and managerial implicati<strong>on</strong>s are discussed<br />

below. Although both paths associated with collaborati<strong>on</strong> and analytics are significantly positive, in the<br />

Danish sample collaborati<strong>on</strong> is much str<strong>on</strong>ger, whereas analytics is much str<strong>on</strong>ger am<strong>on</strong>g Swedish<br />

SMEs, as shown in Table 2. This difference might be explained by the fact that Swedish SMEs have<br />

been using ERP for a l<strong>on</strong>ger time than Danish SMEs (Table 1). Thus, not taking full potential in exploiting<br />

data analytically, and aligned with Bu<strong>on</strong>anno et al. (2005), ERP starters c<strong>on</strong>fer more value to<br />

collaborati<strong>on</strong> because it is often c<strong>on</strong>nected to the organizati<strong>on</strong>al enhancements. However, the difference<br />

in the samples regarding number of years is not great enough that it al<strong>on</strong>e could explain the difference.<br />

But, if also taken into c<strong>on</strong>siderati<strong>on</strong> that Danish have a higher number of service organizati<strong>on</strong>s<br />

and stipulating that service organizati<strong>on</strong> demands more collaborati<strong>on</strong> it could be that both these<br />

characteristics explain the difference. While the link between ‘ERP use’ and ‘ERP value’ is found to be<br />

not significant, it can be stated that there is no difference between Swedish and Danish SMEs in this<br />

respect.<br />

Furthermore, although competitive pressure, best-practices, efficiency, and compatibility are significant,<br />

their importance differs across Swedish SMEs and Danish SMEs, as shown in Table 2. The underlying<br />

rati<strong>on</strong>ale would be that the number of years using the system influences ‘ERP use’. This c<strong>on</strong>clusi<strong>on</strong><br />

might be explained through cross-country analysis. First, although compatibility and efficiency<br />

have significant paths for both Swedish and Danish SMEs, compatibility shows a negative result for<br />

Danish SMEs. This may also be explained by the weaker relati<strong>on</strong>ship that Danish SMEs gives for the<br />

relati<strong>on</strong>ship between analytics and ERP value. The explanati<strong>on</strong> could be that if analytics is not seen<br />

as that important, it could also be that the linkage between different data sources (compatibility) is<br />

also not seen as an important determinant for ERP use. Hence, technological characteristics such as<br />

compatibility with other hardware and software, and transacti<strong>on</strong>al efficiency depend <strong>on</strong> the system<br />

stability, which requires use over time. As a result, ERPs with less customizati<strong>on</strong> (using standard protocols<br />

and best practices) are more suited to face compatibility issues (Bu<strong>on</strong>anno et al., 2005;<br />

Nicolaou and Bhattacharya, 2006). Sec<strong>on</strong>d, following our predicti<strong>on</strong>s, and in line with the c<strong>on</strong>clusi<strong>on</strong>s<br />

of Bradford and Florin (2003), Kositanurit et al. (2006), and Chang et al. (2011), our findings reveal a<br />

negative effect of system complexity <strong>on</strong> ‘ERP use’. However, it is not statistically significant for Swedish<br />

SMEs. It has been widely believed that complexity of business applicati<strong>on</strong>s is an inhibitor to use,<br />

but from our results we could <strong>on</strong>ly provide evidence for this in the Danish sample. Competitive pressure<br />

is statistically significant for both Swedish and Danish SMEs, but is str<strong>on</strong>ger for Swedish SMEs.<br />

This reveals that competitive pressure is a subject where analytics plays a critical role in gaining business<br />

advantages.<br />

The results have several important implicati<strong>on</strong>s for management. In the first place, as ERP diffuses<br />

through greater use, and becomes a necessity, the competitive pressure infuses IT-enhanced capabilities<br />

such as collaborati<strong>on</strong> and business analytics, which are major sources of ERP value. Our<br />

analysis shows that the relati<strong>on</strong>ship between training and ERP use is not found to be statistically significant,<br />

which is a surprise. However, that could probably be explained by the fact that in both cases<br />

Swedish and Danish SMEs have used ERPs for many years. Best practices shows a str<strong>on</strong>ger linkage<br />

am<strong>on</strong>gst Danish SMEs than am<strong>on</strong>gst Swedish SMEs, as does collaborati<strong>on</strong>. This provides indicati<strong>on</strong>s<br />

for managers <strong>on</strong> what the ERP is used for and could therefore be used for managers as a decisi<strong>on</strong><br />

point when making the decisi<strong>on</strong> <strong>on</strong> what specific ERP soluti<strong>on</strong> to implement. Lastly, our study also offers<br />

implicati<strong>on</strong>s for software makers; ERP complexity, business analytics, and business collaborati<strong>on</strong><br />

functi<strong>on</strong>alities have emerged as important factors for ERP use and value. This paper has some limitati<strong>on</strong>s<br />

that may form the starting point for further research. First, although our study shows evidence<br />

that use and value importance vary across-countries in associati<strong>on</strong> with the number of years using<br />

ERP, we cannot speak empirically about the issue of whether the maturity stages play a role, because<br />

this would require an adopti<strong>on</strong> process life-cycle study. An interesting different directi<strong>on</strong> could be to<br />

study the maturity stages of ERP. Sec<strong>on</strong>d, although data cover several industry types, some biases<br />

may have been introduced; perhaps different industries have different operating characteristics and<br />

envir<strong>on</strong>ments, and the factors related to ERP use and value may differ accordingly (Oliveira and<br />

Martins, 2010a). C<strong>on</strong>sequently, we encourage further studies comparing industries.<br />

5. C<strong>on</strong>clusi<strong>on</strong><br />

Via an empirical study am<strong>on</strong>g Scandinavian SMEs we evaluated a research model for assessing ERP<br />

use and ERP value at the firm level based <strong>on</strong> diffusi<strong>on</strong> <strong>on</strong> innovati<strong>on</strong> and resource-based view theory.<br />

While use and value are usually studied separately, our study proposes that use and value are closely<br />

associated for the post-adopti<strong>on</strong> stages of ERPs. Besides being the first model applied to Scandina-<br />

140


Björn Johanss<strong>on</strong> et al.<br />

vian SMEs, our study c<strong>on</strong>tributes to the literature by adding an internati<strong>on</strong>al dimensi<strong>on</strong> and by moving<br />

bey<strong>on</strong>d dichotomous “adopti<strong>on</strong> versus n<strong>on</strong>-adopti<strong>on</strong>”, linking ERP value creati<strong>on</strong> to collaborati<strong>on</strong> and<br />

analytics. For ‘ERP’ use, our study has examined six DOI determinants; whereas competitive pressure,<br />

best-practices and transacti<strong>on</strong>al efficiency are important to both Swedish and Danish SMEs,<br />

cross-country analysis also shows complexity to be an important inhibitor for ‘ERP use’ am<strong>on</strong>g Danish<br />

SMEs, but not significant am<strong>on</strong>g Swedish SMEs. For ‘ERP value’ our study dem<strong>on</strong>strates that collaborati<strong>on</strong><br />

and analytics c<strong>on</strong>tribute to value creati<strong>on</strong> from ERP. Moreover, our study reveals that while<br />

collaborati<strong>on</strong> is more important for Danish SMEs, analytics is more important for Swedish SMEs. The<br />

main c<strong>on</strong>clusi<strong>on</strong> is that Danish and Swedish SMEs present significant different results despite the fact<br />

that in general they seem to be so similar.<br />

Appendix A: Measures<br />

Resp<strong>on</strong>dents were asked to rate their percepti<strong>on</strong> of…<br />

C<strong>on</strong>struct /Items Scale<br />

Compatibility (Bradford and Florin, 2003):<br />

CB1 …with others’ software. 1~5<br />

CB2 …with others’ hardware. 1~5<br />

Complexity (reverse code) (Kositanurit et al., 2006):<br />

CX1 …intuitiveness of the system. 1~5<br />

CX2 …how comfortable users feel using it. 1~5<br />

Efficiency (Rajagopal, 2002):<br />

EF1 …effectiveness in executing repetitive tasks. 1~5<br />

EF2 …effectiveness of user interface. 1~5<br />

EF3 …speed and reliability of system. 1~5<br />

Best-Practices (Wenrich and Ahmad, (2009):<br />

BP1 …set up of the applicati<strong>on</strong>. 1~5<br />

BP2 …map workflows based <strong>on</strong> local requirements. 1~5<br />

BP3 …system adaptability to business needs. 1~5<br />

Training (Bradford and Florin, 2003):<br />

TN1 …understanding of the c<strong>on</strong>tent training material. 1~5<br />

TN2 …applied to daily tasks. 1~5<br />

Competitive Pressure (Oliveira and Martins, 2010b):<br />

CP1 …experienced competitive pressure to use ERP. 1~5<br />

CP2 …firm’s competitors affects your landscape market. 1~5<br />

ERP Use (Zhu and Kraemer, 2005):<br />

ERPU1 …how much time per day do employees work with the system? %<br />

ERPU2 …how many reports are generated per day? %<br />

Collaborati<strong>on</strong> (Gattiker and Goodhue, 2005):<br />

CO1 …collaborate with colleagues. 1~5<br />

CO2 …collaborate with the system. 1~5<br />

CO3 …communicate with suppliers, partners, and customers. 1~5<br />

Analytics (Ruivo and Neto, 2010):<br />

AN1 …comprehensive reporting. 1~5<br />

AN2 …real-time access to informati<strong>on</strong>. 1~5<br />

AN3 …data visibility across departments. 1~5<br />

ERP Value (Zhu and Kraemer, 2005):<br />

ERPV1 …user satisfacti<strong>on</strong>. 1~5<br />

ERPV2 …individual productivity. 1~5<br />

ERPV3 …customer satisfacti<strong>on</strong>. 1~5<br />

ERPV4 …management c<strong>on</strong>trol. 1~5<br />

References<br />

Antero, M. and Riis, P. H. (2011) Strategic <strong>Management</strong> of Network Resources: A Case Study of an ERP<br />

Ecosystem, Internati<strong>on</strong>al Journal of Enterprise Informati<strong>on</strong> Systems, Vol 7 No. 2, pp. 18-33.<br />

Bendoly, E. and Kaefer, F. (2004) Business technology complementarities: impacts of the presence and strategic<br />

timing of ERP <strong>on</strong> B2B e-commerce technology efficiencies, Omega, Vol 32 No. 5, pp. 395-405.<br />

Bharadwaj, A. S. (2000) A resource-based perspective <strong>on</strong> informati<strong>on</strong> technology capability and firm<br />

performance: An empirical investigati<strong>on</strong>, MIS Quarterly, Vol 24 No. 1, pp. 169-196.<br />

Bradford, M. and Florin, J. (2003) Examining the role of innovati<strong>on</strong> diffusi<strong>on</strong> factors <strong>on</strong> the implementati<strong>on</strong><br />

success of enterprise resource planning systems, Internati<strong>on</strong>al Journal of Accounting Informati<strong>on</strong> Systems,<br />

Internati<strong>on</strong>al Journal of Accounting Informati<strong>on</strong> Systems, Vol 4 No. 3, pp. 205-225.<br />

141


Björn Johanss<strong>on</strong> et al.<br />

Bu<strong>on</strong>anno, G., Faverio, P., Pigni, F., Ravarini, A., Sciuto, D. and Tagliavin, M. (2005) Factors affecting ERP<br />

system adopti<strong>on</strong>: A comparative analysis between SMEs and large companies, Journal of Enterprise<br />

Informati<strong>on</strong> <strong>Management</strong>,, Vol 18 No. 1, pp. 384-426.<br />

Calisir, F. and Calisir, F. (2004) The Relati<strong>on</strong> of Interface Usability Characteristics, Perceived Usefulness, and<br />

Perceived Ease of Use to End-User Satisfacti<strong>on</strong> with Enterprise Resource Planning (ERP) Systems,<br />

Computers in Human Behavior, Vol 20 No. 4, pp. 505-515.<br />

Chang, H.-H., Chou, H.-W., Lin, C.-P. Y. and Cecilia, I. (2011) ERP Post-Implementati<strong>on</strong> Learning, ERP Usage<br />

and Individual Performance Impact, 15th Pacific Asia <str<strong>on</strong>g>C<strong>on</strong>ference</str<strong>on</strong>g> <strong>on</strong> Informati<strong>on</strong> Systems, Vol. 35<br />

Brisbane, Australia.<br />

Chiang, A. (2009) Creating Dashboards: The Players and Collaborati<strong>on</strong> You Need for a Successful Project,<br />

Business Intelligence Journal, Vol 14 No. 1, pp. 59-63.<br />

Chin, W. W. (1998) Issues and Opini<strong>on</strong> <strong>on</strong> Stuctural Equati<strong>on</strong> Modeling, MIS Quarterly, Vol 22 No. 1, pp. 7-16.<br />

Chou, S. W. and Chang, Y. C. (2008) The implementati<strong>on</strong> factors that influence the ERP (enterprise resource<br />

planning) benefits, Decisi<strong>on</strong> Support Systems, Vol 46 No. 1, pp. 149-157.<br />

Cooper, R. and Zmud, R. (1990) Informati<strong>on</strong> Technology Implementati<strong>on</strong> Research: A Technological Diffusi<strong>on</strong><br />

Approach, <strong>Management</strong> Science, Vol 3 No. 2, pp. 123-139.<br />

Davenport, T. H. (1998) Putting the enterprise into the enterprise system, Harvard Business Review, Vol 76 No.<br />

4, pp. 121-131.<br />

Davenport, T. H. and Harris, J. G. (2007) Competing <strong>on</strong> Analytics: The New Science of Winning, Harvard<br />

Business School Press.<br />

Devaraj, S. and Kohli, R. (2003) Performance impacts of informati<strong>on</strong> technology: Is actual usage the missing<br />

link?, <strong>Management</strong> Science, Vol 49 No. 3, pp. 273–289.<br />

Elbertsen, L., Benders, J. and Nijssen, E. (2006) ERP use: exclusive or complemented?, Industrial <strong>Management</strong><br />

& Data Systems, Vol 106 No. 6, pp. 811-824.<br />

Fosser, E., Leister, O. H., Moe, C. E. and Newman, M. (2008) Organizati<strong>on</strong>s and vanilla software: What do we<br />

know about ERP systems and competitive advantage?, <str<strong>on</strong>g>European</str<strong>on</strong>g> c<strong>on</strong>ference <strong>on</strong> informati<strong>on</strong> systems, Vol.<br />

8 (Ed, ECIS) Galway, Irland, pp. 2460–2471.<br />

Gattiker, T. F. and Goodhue, D. L. (2005) What happens after ERP implementati<strong>on</strong> <strong>on</strong> plant-level outcomes, MIS<br />

Quarterly, Vol 29 No. 3, pp. 559-585.<br />

Hair, J., Anders<strong>on</strong>, R., Tatham, R. and Black, W. (1998) Multivariate Data Analysis, Prentice Hall, New Jersey.<br />

Hedman, J. and Kalling, T. (2003) The business model c<strong>on</strong>cept: theoretical underpinnings and empirical<br />

illustrati<strong>on</strong>s, <str<strong>on</strong>g>European</str<strong>on</strong>g> Journal of Informati<strong>on</strong> Systems, Vol 12 No. 49-59.<br />

Kositanurit, B., Ngwenyama, O. and Osei-Brys<strong>on</strong>, K. (2006) An explorati<strong>on</strong> of factors that impact individual<br />

performance in an ERP envir<strong>on</strong>ment: An analysis using multiple analytical techniques, <str<strong>on</strong>g>European</str<strong>on</strong>g> Journal of<br />

Informati<strong>on</strong> Systems, Vol 15 No. 556-568.<br />

Maguire, S., Ojiako, U. and Said, A. (2010) ERP implementati<strong>on</strong> in Omantel: a case study, Industrial<br />

<strong>Management</strong> & Data Systems, Vol 110 No. 1, pp. 78-92.<br />

Mata, F. J., Fuerst, W. L. and Barney, J. B. (1995) Informati<strong>on</strong> technology and sustained competitive advantage:<br />

A resource-based analysis, MIS Quarterly, Vol 9 No. 4, pp. 487–505.<br />

Nah, F., Tan, X. and Teh, S. H. (2004) An Investigati<strong>on</strong> <strong>on</strong> End-Users Acceptance of Enterprise Systems,<br />

Informati<strong>on</strong> Resources <strong>Management</strong> Journal, Vol 17 No. 3, pp. 32-53.<br />

Nicolaou, A. and Bhattacharya, S. (2006) Organizati<strong>on</strong>al performance effects of ERP systems usage: the impact<br />

of post implementati<strong>on</strong> changes, Internati<strong>on</strong>al Journal of Accounting Informati<strong>on</strong> Systems, Vol 7 No. 18–35.<br />

O’Leary, D. (2000) Enterprise resource planning: systems, life cycle, electr<strong>on</strong>ic commerce, and risk, Univ. Press,<br />

New York: Cambridge.<br />

Oliveira, T. and Martins, M. F. (2010a) Firms Patterns of E-business Adopti<strong>on</strong>: Evidence for the <str<strong>on</strong>g>European</str<strong>on</strong>g> Uni<strong>on</strong>-<br />

27, The Electr<strong>on</strong>ic Journal Informati<strong>on</strong> Systems Evaluati<strong>on</strong>, Vol 13 No. 1, pp. 47-56.<br />

Oliveira, T. and Martins, M. F. (2010b) Understanding e-business adopti<strong>on</strong> across industries in <str<strong>on</strong>g>European</str<strong>on</strong>g><br />

countries, Industrial <strong>Management</strong> & Data Systems, Vol 110 No. 9, pp. 1337-1354.<br />

Rajagopal, P. (2002) An innovati<strong>on</strong>-diffusi<strong>on</strong> view of implementati<strong>on</strong> of enterprise resource planning (ERP)<br />

systems and development of a research model, Informati<strong>on</strong> & <strong>Management</strong>, Vol 40 No. 87-114, pp.<br />

Rogers, E. M. (1995) Diffusi<strong>on</strong> of innovati<strong>on</strong>s, The Free Press, New York.<br />

Ruivo, P. and Neto, M. (2010) ERP software for SMEs in Portugal: Exploratory study of new KPIs, 4th <str<strong>on</strong>g>European</str<strong>on</strong>g><br />

<str<strong>on</strong>g>C<strong>on</strong>ference</str<strong>on</strong>g> <strong>on</strong> Informati<strong>on</strong> <strong>Management</strong> and Evaluati<strong>on</strong>Lisb<strong>on</strong>, Portugal, pp. 421-428.<br />

Ruivo, P., Oliveira, T. and Neto, M. (2012) ERP use and value: Portuguese and Spanish SMEs, Industrial<br />

<strong>Management</strong> & Data Systems, Vol 112 No. 7, pp. in press.<br />

Shahin, D. and Ainin, S. (2011) The influence of organizati<strong>on</strong>al factors <strong>on</strong> successful ERP implementati<strong>on</strong>,<br />

<strong>Management</strong> Decisi<strong>on</strong>, Vol 49 No. 6, pp. 911-926.<br />

Wenrich, K. and Ahmad, N. (2009) Less<strong>on</strong>s learned during a decade of ERP experience: A case study,<br />

Internati<strong>on</strong>al Journal of Enterprise Informati<strong>on</strong> Systems, Vol 5 No. 1, pp. 55-73.<br />

Zhu, K. and Kraemer, K. L. (2005) Post-adopti<strong>on</strong> variati<strong>on</strong>s in usage and value of e-business by organizati<strong>on</strong>s:<br />

Cross-country evidence from the retail industry, Informati<strong>on</strong> Systems Research, Vol 16 No. 1, pp. 61-84.<br />

142


User Experience in Mobile Ph<strong>on</strong>es by Using Semantic<br />

Differential Methodology<br />

Kalimullah Khan<br />

School of Computing, Blekinge Institute of Technology, Sweden<br />

Kakh08@bth.student.se<br />

Abstract: Measuring overall UX is a challenging process because of its versatile nature. Studies showed that<br />

hundreds of thousands of products are returned each year not because of its functi<strong>on</strong>al behaviour but because of<br />

its bad user experience. Researchers and practiti<strong>on</strong>ers use different techniques and methods to capture the<br />

customer psychological and behavioural aspects towards a product and to put it into design so that the future<br />

product form must be in according to his/her expectati<strong>on</strong>s. In this paper a research work is carried out to evaluate<br />

user experience evaluati<strong>on</strong> methodologies and to identify a method which can be used efficiently to measure the<br />

overall user experience of a product use from user experience using mobile ph<strong>on</strong>e as a case. As overall user<br />

experience c<strong>on</strong>stitutes both the experiential as well as n<strong>on</strong>- experiential aspects of a product. Hence semantic<br />

differential methodology is identified as a best suited method based <strong>on</strong> current user experience evaluati<strong>on</strong><br />

methodologies and later <strong>on</strong> used to measure preference from the overall user experience which can be used to<br />

improve product form to ensure customer loyalty.<br />

Keywords: semantic differential (SD), user experience (UX), overall user experience, user preferences, UX<br />

metrics<br />

1. Introducti<strong>on</strong><br />

In daily life a pers<strong>on</strong> is encountered with a wide range of products and share different experiences. As<br />

human experience is complex and evolve all the time therefore very difficult to measure. Both<br />

experiential and n<strong>on</strong>-experiential experience make it complex due to which it becomes dynamic,<br />

complex and subjective; therefore the success of technology adheres to experiential as well as the<br />

n<strong>on</strong> experiential aspects.<br />

The authors used semantics covering broadly the overall user experience of technology use such as<br />

social, emoti<strong>on</strong>al, aesthetic, fun, joy, cool and mood etc to measure overall user experience. The<br />

authors’ also claimed that SDM can be used to measure overall user experience in order to identify<br />

user preferences in short time and with less resource c<strong>on</strong>sumpti<strong>on</strong>s. The results obtained through this<br />

methodology resulting in positive experience indicate satisfacti<strong>on</strong> while preferences resulted in<br />

negative experience indicating improvements needed in product form. Preference indicates that<br />

product form can be improved accordingly to achieve satisfacti<strong>on</strong> and loyalty.<br />

2. User experience theory<br />

User experience is ubiquitous. The user churns the experience the moment he start using the<br />

product. As UX is a resultant of behavioural, temporal and psychological aspects. Therefore, it is very<br />

crucial to know UX evaluati<strong>on</strong> and its results <strong>on</strong> product development. Therefore, before starting<br />

evaluating the n<strong>on</strong> experiential aspects of products, <strong>on</strong>e needs to understand the actual meaning of<br />

‘user experience evaluati<strong>on</strong>’. In order to do so, five steps have been c<strong>on</strong>sidered in below paragraphs<br />

to develop a systematic way to evaluate user experience.<br />

User Experience<br />

To gain a comm<strong>on</strong> understanding about UX and make science out of it, we should be able to<br />

formulate a definiti<strong>on</strong> that every<strong>on</strong>e can agree <strong>on</strong>. There are a number of UX definiti<strong>on</strong>s out there;<br />

below are few of them.<br />

“All the aspects of how people use an interactive product: the way it feels in their hands, how well they<br />

understand how it works, how they feel about it while they’re using it, how well it serves their<br />

purposes, and how well it fits into the entire c<strong>on</strong>text in which they are using it” (Hassenzahl &<br />

Tractinsky, 2006). This definiti<strong>on</strong> fits well according to the overall UX. It defines their identity factor,<br />

“the way it feels in their hands”, aesthetics, “how they feel about it while they’re using it”, utility, “how<br />

well it serves their purpose”, stimulati<strong>on</strong> “how well it fits into the entire c<strong>on</strong>text in which they are using<br />

it”.<br />

143


Kalimullah Khan<br />

Other definiti<strong>on</strong>s which are suitable according to overall UX aspects are:<br />

Kuniavsky believes that the precise definiti<strong>on</strong> of user experience is very difficult; because user<br />

interacts with others and in return the user experience is made omnipresent by the envir<strong>on</strong>ment<br />

(Kuniavsky, 2003). Internati<strong>on</strong>al Standard Organizati<strong>on</strong> (ISO) has its own view of defining user<br />

experience. According to the current versi<strong>on</strong>, it is defined as “a pers<strong>on</strong>’s percepti<strong>on</strong>s and resp<strong>on</strong>ses<br />

that result from the use of anticipated use of a product, system or service” (ISO, 2008). Similarly,<br />

according to Usability Professi<strong>on</strong>als Associati<strong>on</strong> (2007), “Every aspect of the user’s interacti<strong>on</strong> with a<br />

product, service, or company that makes up the user’s percepti<strong>on</strong>s of the whole”. Logan et al (1994)<br />

argued for the importance of whole range of specific n<strong>on</strong> pragmatic needs such as surprise, diversi<strong>on</strong><br />

or intimacy to be addressed by technology. “Drawing up<strong>on</strong> the c<strong>on</strong>cept of emoti<strong>on</strong>al usability”.<br />

Hassenzahl asserts that future HCI must be c<strong>on</strong>cerned about the pragmatic aspects of interactive<br />

products as well as about hed<strong>on</strong>ic aspect, such as stimulati<strong>on</strong>, identificati<strong>on</strong> and evocati<strong>on</strong> so that his<br />

multidimensi<strong>on</strong>al model explicitly links product attributes with needs and values that could facilitate in<br />

the evaluati<strong>on</strong> of UX (Hassenzahl, 2003). Similarly, according to Mäkelä et al (2001), “Result of<br />

motivated acti<strong>on</strong> in a certain c<strong>on</strong>text”. “The user experience c<strong>on</strong>siders the wider relati<strong>on</strong>ship between<br />

the product and the user in order to investigate the individual’s pers<strong>on</strong>al experience of using it”<br />

(Hassenzahl, 2006).<br />

Comp<strong>on</strong>ents of User Experience<br />

The aim of measuring user experience is to c<strong>on</strong>sider a more comprehensive user centred approach<br />

that takes also aspects into account which go bey<strong>on</strong>d usability. One approach for defining the<br />

comp<strong>on</strong>ents of user experience is to characterize specific dimensi<strong>on</strong>s that are important aspects in<br />

the experience of a product. For this purpose, Hassenzahl, distinguishes two dimensi<strong>on</strong>s of product<br />

qualities, namely the percepti<strong>on</strong> of instrumental (or: ‘pragmatic”) and n<strong>on</strong>-instrumental (or: ‘hed<strong>on</strong>ic”)<br />

qualities (Jordan, 2000). The importance of those aspects is motivated by their immediate<br />

understandability. While usability evaluati<strong>on</strong> depends basically <strong>on</strong> interacti<strong>on</strong> with the product, the<br />

attributes that enable hed<strong>on</strong>ic judgments are inherent in the product appearance itself.<br />

A third important aspect of user experience is emoti<strong>on</strong>al user reacti<strong>on</strong>s. For example, in his<br />

hierarchical model, Jordan distinguished several types of pleasure with a product, whereby he insists<br />

<strong>on</strong> high functi<strong>on</strong>ality and high usability as necessary prec<strong>on</strong>diti<strong>on</strong>s (Karapanos et al, 2008).<br />

User experience metrics<br />

UX is a subjective and holistic c<strong>on</strong>cept; it is not easy to define a set of criteria against which it could<br />

be evaluated, but there are user experience evaluati<strong>on</strong>s where the participants evaluate the product<br />

against the pers<strong>on</strong>al criteria (Norman, 2009). This way of metrics setting is very interesting as it well<br />

addresses the subjective nature of user experience.<br />

From product creati<strong>on</strong> perspective, each product aims at certain user experience, e.g. fun, trust, or<br />

relaxing. In this case, it is useful to define metrics from the product’s perspective rather than from<br />

each individual’s perspective. UX is more dynamic therefore it is very hard to define its metrics. Some<br />

practiti<strong>on</strong>ers evaluate aesthetics while others look for emoti<strong>on</strong>al aspects. Hence UX can be<br />

momentary, episodic and l<strong>on</strong>g-term adding value to product design.<br />

User experience evaluati<strong>on</strong> methods<br />

UX is a complex c<strong>on</strong>cept that requires specific evaluati<strong>on</strong> techniques in order to c<strong>on</strong>sider its all<br />

aspects. These techniques can be very resource c<strong>on</strong>suming, including people, time, and m<strong>on</strong>ey.<br />

Therefore cost-effective evaluati<strong>on</strong> techniques are of a great importance.<br />

Numerous potential criteria for applicable methods are obvious from literature; therefore <strong>on</strong>e method<br />

will not serve all purposes (Osgood, 1952). A set of methods should form a “User experience<br />

evaluati<strong>on</strong> tool kit” so that user experience practiti<strong>on</strong>ers can select an applicable method for each<br />

case. There are methods developed for examining user’s momentary emoti<strong>on</strong>s during interacting with<br />

a system (Chang et al, 2002) or for analysing the emoti<strong>on</strong>s after interacti<strong>on</strong> (Guo, 2010). In user<br />

experience literature, a movement from emoti<strong>on</strong> assessment towards a l<strong>on</strong>ger period of time has also<br />

been noticed (Azhari, 2007).This movement means a change in the way user experience is evaluated<br />

144


Kalimullah Khan<br />

and <strong>on</strong>e should not <strong>on</strong>ly investigate momentary emoti<strong>on</strong>s but also examine how users experience a<br />

product as a whole during a l<strong>on</strong>ger period which results in satisfacti<strong>on</strong>.<br />

There exist a number of methodologies to evaluate n<strong>on</strong>-experiential aspects of technology use in a<br />

particular c<strong>on</strong>text with particular system and mode of use but no specific method found in the<br />

literature to measure overall user experience. From the literature review we came across various<br />

UXEMS but they can be used as momentary, episodic or l<strong>on</strong>g-term. They are highly time c<strong>on</strong>suming<br />

and difficult to run emoti<strong>on</strong> analysis. They require high skills to evaluate user experience and difficult<br />

to interpret user results. Some of methodologies found after c<strong>on</strong>ducting literature review are as follow.<br />

Repertory Grid Technique<br />

Experimental Pilots<br />

SAFE Method<br />

Web Based Surveys<br />

Diary Method<br />

Interviews<br />

Heuristic Evaluati<strong>on</strong> Methods<br />

Cognitive Walkthrough<br />

Integrate user experience evaluati<strong>on</strong> into product development<br />

The basic user-centred design principles apply for user experience design as well. We should first<br />

c<strong>on</strong>sider users’ needs and wants in selected c<strong>on</strong>texts, then iteratively design and evaluate the<br />

c<strong>on</strong>cepts during the product development process. The methods are different in different phases of<br />

product development: in the very early phases, c<strong>on</strong>cept ideas can be evaluated with surveys.<br />

Resource-efficient user experience evaluati<strong>on</strong> methods help introducing user studies as new activities<br />

into the companies that have not followed user-centred design earlier. The goal of user evaluati<strong>on</strong> is<br />

to ensure that all products will be valuable and enjoyable for the target users, and this pay backs in<br />

customer satisfacti<strong>on</strong> to ensure customer loyalty.<br />

3. Factor analysis<br />

Thurst<strong>on</strong>e (1947) was the originator of factor analysis, which was developed in the area of<br />

psychometrics. Nowadays, the method is frequently used as a statistical data reducti<strong>on</strong> technique to<br />

explain variability am<strong>on</strong>g observed random variables in terms of fewer unobserved random variables<br />

called factors. The technique is used in many fields, including ec<strong>on</strong>omics and sociology. The method<br />

can now be implemented easily using c<strong>on</strong>venient software packages, even by users who lack detailed<br />

knowledge of mathematical background. Factor Analysis is normally used to identify underlying<br />

communalities am<strong>on</strong>gst the scales employed. The most frequently obtained communalities or factors<br />

are.<br />

Evaluati<strong>on</strong>, defined by adjectives such as liked- disliked, positive- negative, h<strong>on</strong>est- dish<strong>on</strong>est,<br />

Potency, defined by heavy- light, str<strong>on</strong>g- weak, hard- soft.<br />

Activity, defined by adjectives such as active- passive, hot- cold, fast- slow.<br />

4. Semantic differential methodology<br />

SDM is recognized as a useful method for measuring pers<strong>on</strong>’s semantic images of a c<strong>on</strong>cept, and<br />

many examples have been reported, applied in various areas. It has proven to be flexible and reliable<br />

instrument for measuring attitudes to a wide range of stimuli. The method normally employs rating of<br />

stimuli by using bipolar scales. Each bipolar scale is defined by a pair of adjectives with c<strong>on</strong>trasting<br />

meanings such as Fast- Slow, Cheap- expensive.<br />

Semantic differential is a method use to evaluate overall user experience not <strong>on</strong>ly in a short period of<br />

time with less resource c<strong>on</strong>sumpti<strong>on</strong>s but also produce satisfactory results. Azhari (2007) highlighted<br />

the importance of the methodology of market research and feedback with known demographic profiles<br />

for better design using semantic differential methodology with multidimensi<strong>on</strong>al scaling approach. The<br />

study was c<strong>on</strong>ducted in two different countries, Australia and Malaysia. It was claimed that the degree<br />

of satisfacti<strong>on</strong> towards a particular product lies in the perceived value of users of different age groups.<br />

145


Kalimullah Khan<br />

Users of different age groups behave differently towards the same product use, because there exist<br />

differences in the c<strong>on</strong>sumer’s behaviour. In this study SD method was used to evaluate the<br />

preferences and image percepti<strong>on</strong>s of mobile ph<strong>on</strong>es. In another study Green et al (2009) identify the<br />

parameters of customer satisfacti<strong>on</strong> and loyalty for mobile ph<strong>on</strong>es. Impact of perceived customer<br />

value, perceived service quality, trust <strong>on</strong> customer and the influence of satisfacti<strong>on</strong> <strong>on</strong> loyalty was<br />

investigated. A questi<strong>on</strong>naire survey was used to collect data customer preferences <strong>on</strong> above<br />

menti<strong>on</strong>ed artifacts. It was found that customer satisfacti<strong>on</strong> and loyalty is predicted by trust and<br />

emoti<strong>on</strong>al value. It was also further c<strong>on</strong>cluded that perceived service quality is a significant factor<br />

influencing trust which affect customer satisfacti<strong>on</strong> and cause changes up to an extent in loyalty.<br />

In this paper the authors used semantics covering broadly the overall user experience of technology<br />

use such as social, emoti<strong>on</strong>al, aesthetic, fun, joy, cool and mood etc to measure overall user<br />

experience. It is also claimed that SDM can be used to measure overall user experience in order to<br />

identify user preferences in short time and with less resource c<strong>on</strong>sumpti<strong>on</strong>s. The results obtained<br />

through SDM resulting in positive experience indicate satisfacti<strong>on</strong> while preferences resulted in<br />

negative experience indicating improvements needed in product form. Preference indicates that<br />

product form can be improved accordingly to achieve satisfacti<strong>on</strong> and loyalty.<br />

5. Research methodology<br />

Research methodology is a plan or strategy to c<strong>on</strong>duct a research work in a scientific way and link<br />

methods to outcome (Creswell, 2002). It defines how to develop research activity and what<br />

measurement should be used to advance the research and achieve research goals.<br />

In this paper the work presented is the c<strong>on</strong>fluence of two approaches and answers to the problems<br />

are sought through the following methodologies.<br />

Literature review<br />

Survey<br />

In first phase, detail explorative literature review is c<strong>on</strong>ducted; from literature review the relevant<br />

latent c<strong>on</strong>structs which affect customer loyalty are identified. The identified c<strong>on</strong>structs are based <strong>on</strong><br />

user experience. The role of SD in UX studies to achieve customer satisfacti<strong>on</strong> and loyalty is<br />

identified. First phase is helpful to know the empirical evidence of Semantic Differential Methodology.<br />

In sec<strong>on</strong>d phase, the author used data collecti<strong>on</strong> method questi<strong>on</strong>naire (survey) with the students of<br />

BTH (Blekinge Institute of Technology) to understand their experience of using mobiles to measure<br />

overall UX. The questi<strong>on</strong>naire is based <strong>on</strong> semantic differentials of bipolar pairs of adjectives. Initially<br />

all possible pair of adjectives are selected which best explain psychological aspects of customers,<br />

pragmatic and hed<strong>on</strong>ic qualities of mobile ph<strong>on</strong>e products and they were selected from literature used<br />

explicitly for mobile ph<strong>on</strong>e products evaluati<strong>on</strong>. These pair of adjectives served as characteristics or<br />

attributes which best explain user’s percepti<strong>on</strong> and overall UX of customers from a mobile ph<strong>on</strong>e<br />

product. A survey is c<strong>on</strong>ducted in order to collect user evaluati<strong>on</strong> data <strong>on</strong> mobile ph<strong>on</strong>es by using SD<br />

method.<br />

After getting user evaluati<strong>on</strong> data, factor analysis has been c<strong>on</strong>ducted to identify best possible factors<br />

which show customer preference coming from overall UX of mobile ph<strong>on</strong>e products. Questi<strong>on</strong>naire<br />

significance is tested statistically through Cr<strong>on</strong>bach's alpha value (Cr<strong>on</strong>bach, 1951). For research<br />

purposes alpha should be more than 0.7 to 0.8. Models have been c<strong>on</strong>structed by author using<br />

stepwise regressi<strong>on</strong> analysis of customer preferences for mobile ph<strong>on</strong>es. The models shows a str<strong>on</strong>g<br />

relati<strong>on</strong>ship of the customer preferences to mobile ph<strong>on</strong>e experience and carries an explanati<strong>on</strong> of<br />

various parameters of experience that if implemented in design, the future mobile ph<strong>on</strong>e product will<br />

not <strong>on</strong>ly exhibit satisfacti<strong>on</strong> but also increase profitability and will ensure loyalty.<br />

6. Results<br />

The author mainly c<strong>on</strong>cluded from LR results that SDM can be used in c<strong>on</strong>juncti<strong>on</strong> with satisfacti<strong>on</strong> in<br />

order to evaluate overall UX and to identify user preference. Literature review results indicate that<br />

SDM is used in evaluati<strong>on</strong> of UX in different c<strong>on</strong>texts. Therefore, author used this method as a<br />

method of investigati<strong>on</strong> to identify the role of SD to evaluate pragmatic and hed<strong>on</strong>ic aspects of UX<br />

al<strong>on</strong>g with measures of the UX. The author identified satisfacti<strong>on</strong> being used as a measure to<br />

evaluate UX as 72% as show in Table 1. The selected papers were studied and 16 out of 22 provide<br />

146


Kalimullah Khan<br />

the evidence of satisfacti<strong>on</strong> as a measure to evaluate UX. 13 out of 22 papers provide the evidence of<br />

hed<strong>on</strong>ic factors that is 59% while 11 out of 22 papers shows pragmatic use i.e. 50%. Other measures<br />

of UX; emoti<strong>on</strong> and expectati<strong>on</strong> were 45% respectively identified.<br />

Table 1: Total number of occurrences of factors/ Methodology<br />

Factor/ Method of Investigati<strong>on</strong> Number of Occurrences Percentage<br />

Satisfacti<strong>on</strong> 16/22 72<br />

Hed<strong>on</strong>ic 13/22 59<br />

Pragmatic 11/22 50<br />

Emoti<strong>on</strong> 10/22 45<br />

Expectati<strong>on</strong> 10/22 45<br />

Sec<strong>on</strong>d phase is based <strong>on</strong> empirical study c<strong>on</strong>ducti<strong>on</strong>. This phase mainly served as to put the<br />

informati<strong>on</strong> collected from first phase into practical example in order to identify user preferences<br />

through SDM. To do so, initially 52 different semantics of factors like aesthetics, emoti<strong>on</strong>s, identity,<br />

utility and stimulati<strong>on</strong> were selected from literature for collecting UX data. Semantics are collected<br />

from literature used in different studies to evaluate user experience. The semantics were grouped<br />

together according to Osgood distributi<strong>on</strong> of bipolar adjectives Activity, Evaluati<strong>on</strong> and Potency<br />

accordingly (Osgood, 1952). Pilot survey, was c<strong>on</strong>ducted in order to define scale and finally 31<br />

semantics were selected for final data collecti<strong>on</strong> as shown in Table II below.<br />

Table 2: Distributi<strong>on</strong> of adjectives according to Osgood [13]<br />

Activity Potency Evaluati<strong>on</strong><br />

Adjective Ant<strong>on</strong>ym Adjective Ant<strong>on</strong>ym Adjective Ant<strong>on</strong>ym<br />

Practical Decorative Unadomed Splendid Traditi<strong>on</strong>al Modern<br />

Adoring Practical Masculine Feminine Popular Individual<br />

Geometrical Streamline Complex C<strong>on</strong>cise Hand-made Hi-tech<br />

Abrupt Unis<strong>on</strong>ous Inc<strong>on</strong>venient C<strong>on</strong>venient Rough Delicate<br />

Pointed Rounded Babyish Mature Mediocre Noble<br />

Hale Fluid Dull Captivating Heavy Light<br />

Cheap Expensive Tardy Streamlined Clumsy Clever<br />

Normal Particular Sharp-edge Curvature Cautious Bold<br />

C<strong>on</strong>venti<strong>on</strong>al Inventive Repelling Appealing Coarse Delicate<br />

Undemanding Challenging Indistinct Distinct Rejecting Inviting<br />

Disagreeable Agreeable<br />

SD data collecti<strong>on</strong> sheet was used to collect data. Data was collected from 21 participants’ mainly<br />

BTH (Blekinge Institute of Technology) students for further analysis. A statistical technique named<br />

factor analysis was employed in order to form groups of semantics exhibiting particular<br />

characteristics. Each group is formed <strong>on</strong> the basis of likelihood of characteristics of same semantics<br />

of various factors. There are 9 factors whose Eigenvalue is greater than <strong>on</strong>e, but <strong>on</strong>ly first four factors<br />

were selected as the remaining factors have very few semantics to be c<strong>on</strong>sidered for further analysis<br />

as shown in Table III and figure no 1 respectively.<br />

Figure 1: Factor analysis chart<br />

147


Kalimullah Khan<br />

Above Figure no 1 represents the semantic representati<strong>on</strong> of the factors identified in UX through<br />

SDM. The average values of the four factors for the evaluated product are also plotted in the above<br />

figure. F1 represents positive experience for Pragmaticility factor. While F2 (Practicality) is neutral.<br />

Whereas F3 (Activity) is turned out to be negative from UX perspectives. F4 (Structural) represents<br />

positive experience in UX; showing that the product has good shape but need improvement.<br />

Table 3: Factors with Eigenvalue greater than 1<br />

Eigen values<br />

F1 F2 F3 F4 F5 F6 F7 F8 F9<br />

Eigen values .470 .867 .349 .126 .036 .757 .324 1.185 .043<br />

Variability (%) 4.097 2.475 0.803 0.085 .569 .668 .272 .823 3.366<br />

Cumulative (%) 4.097 6.573 7.376 7.461 4.030 69.697 73.969 77.792 81.158<br />

There are 17 factors but <strong>on</strong>ly nine factors that the initial Eigenvalue was over 1 and whose Extracti<strong>on</strong><br />

Sums of Squared Loadings was up to 81.158 were selected as shown in above Table III . It could be<br />

further interpreted in figure no 2 below.<br />

Figure 2: Eigenvalues plot<br />

Based <strong>on</strong> factor pattern analysis and the underlying communalities of adjectives in each factor, four<br />

factors were selected for final analysis and that is the reas<strong>on</strong> being very less number of variables in<br />

particular factor categories used. Factor loadings, where Eigenvalues greater than 1 extracted from<br />

final data set as shown in figure no 2. The semantic differential chart computed from the final survey<br />

is provided in figure no 3. The mean values of the opposite pairs of adjectives are computed. Extreme<br />

values are of particular interest. The extreme values give an idea that which characteristics are<br />

particularly critical and need more improvement or particularly well resolved.<br />

Figure 3: Semantic differential chart<br />

148


Kalimullah Khan<br />

Figure no 3 represents that overall UX of mobile ph<strong>on</strong>es from usability point of view and functi<strong>on</strong>ality<br />

wise is satisfactory and showed positive experience. While from practicality point of view there is more<br />

negative experience observed. The experience showed that mobile ph<strong>on</strong>e from should be in a way<br />

which is more practical, c<strong>on</strong>venient and should be simple and normal. While from activity and<br />

structural wise more negative experience is observed.<br />

The semantic differential chart in Figure no 3 computed from SD data represents the customer<br />

aesthetics, expectati<strong>on</strong>s, emoti<strong>on</strong> and identity. The positive values <strong>on</strong> scale represent customer<br />

satisfacti<strong>on</strong> while negative needs improvement.<br />

7. C<strong>on</strong>clusi<strong>on</strong><br />

The main purpose of this paper is to propose a methodology that is used to measure overall UX to<br />

explore various user preferences suitable for product form improvement. Semantics of various factors<br />

(aesthetics, emoti<strong>on</strong>s, expectati<strong>on</strong>s, fun, and joy etc) were used to measure overall UX. A set of<br />

product semantics al<strong>on</strong>g with user preferences were identified. To improve product from it is<br />

necessary that it should not <strong>on</strong>ly satisfy functi<strong>on</strong>al requirements but it should also satisfy customers’<br />

psychological needs subjectively to guide users with life experience. Various methodologies and tools<br />

used to measure UX were studied in the light of their strengths and weaknesses. There is no<br />

evidence found in literature about a specific user experience methodology to measure overall UX. The<br />

importance of n<strong>on</strong>-experiential perspective of technology use and its worthiness in design is explored.<br />

Product specific semantics used to measure overall UX were identified. It is found that there is no<br />

definite set of metrics which can be used to measure UX rather it depends <strong>on</strong> study type and<br />

envir<strong>on</strong>ment. Furthermore, satisfacti<strong>on</strong> was used as main metric to measure overall UX. Satisfacti<strong>on</strong><br />

was measured using various factors semantics such as emoti<strong>on</strong>s, aesthetics and expectati<strong>on</strong> etc.<br />

8. Future work<br />

SDM was used to measure overall UX in order to identify user preferences. Semantic chart provides<br />

an explanati<strong>on</strong> of the weak design form of the product. It is suggested that SDM should be compared<br />

with other self reporting techniques and observe its effectiveness.<br />

References<br />

Azhari bin Mohd Hashim, Raja, A., (2007), “The multidimensi<strong>on</strong>al Scaling: an interactive method for establishing<br />

percepti<strong>on</strong>s of the appearance of product”.<br />

Cr<strong>on</strong>bach, L. J., (1951), Coefficient alpha and the internal structure of tests. Psychometric Vol. 16, pp.297-334.<br />

Creswell, J.W., (2002), “Research Design – Qualitative, Quantitative and Mixed Method Approaches”, Sage<br />

Publicati<strong>on</strong>s, 2nd ed., ISBN 0-7619-2442-6.<br />

Chang, Chien-Cheng. Shih, Yi. Yen., (2002), “A differential study <strong>on</strong> the product form percepti<strong>on</strong>s of different age<br />

group users”.<br />

Green, W et al., (2008), "Developing the scale adopti<strong>on</strong> framework for evaluati<strong>on</strong> (SAFE)," pp. 49.<br />

Guo. Fu. Tian., (2010), “C<strong>on</strong>sumer demand oriented study <strong>on</strong> mobile ph<strong>on</strong>es’ form percepti<strong>on</strong> design method”,<br />

IEEE.<br />

Hassenzahl. M., (2006). Hed<strong>on</strong>ic, emoti<strong>on</strong>al and experiential perspectives <strong>on</strong> product quality In: C. Ghaoui (Ed.),<br />

Encyclopedia of Human Computer Interacti<strong>on</strong>, pp.266-272.<br />

Hassenzahl, M. and Tractinsky, N., (2006), User experience - a research agenda: Behavior & Informati<strong>on</strong><br />

Technology, Vol 25, No. 2, pp. 91-97.<br />

Hassenzahl. M., (2003), The thing and I: understanding the relati<strong>on</strong>ship between user and product. In Funology:<br />

From Usability to Enjoyment.<br />

ISO DIS 9241-210: (2008), Erg<strong>on</strong>omics of human system interacti<strong>on</strong> - Part 210: Human-centered design for<br />

interactive systems (formerly known as 13407). Internati<strong>on</strong>al organizati<strong>on</strong> for standardizati<strong>on</strong> (ISO).<br />

Switzerland.<br />

Jordan, P. W., (2000), Designing Pleasurable Products: An Introducti<strong>on</strong> to New Human Factors. Taylor &<br />

Francis, L<strong>on</strong>d<strong>on</strong>.<br />

Kuniavsky, M., (2003), observing the user experience: a practiti<strong>on</strong>er's guide to user research: Morgan Kaufmann.<br />

Karapanos. E., Zimmerman. J. Forlizzi. J. Jean-Bernard Martens., (2010), measuring the dynamics of<br />

remembered experience over time: Eindhoven University of Technology, Department of Industrial Design,<br />

Carnegie Mell<strong>on</strong> University, Human-Computer Interacti<strong>on</strong> Institute and School of Design, Pittsburg, PA<br />

15213, USA.<br />

Logan, R.J., Augaitis. S. and Renk. T., (1994), Design of simplified televisi<strong>on</strong> remote c<strong>on</strong>trols: a case for<br />

behavioral and emoti<strong>on</strong>al usability. In Proceedings of the 38th Human Factors and Erg<strong>on</strong>omics Society<br />

Annual Meeting (Santa M<strong>on</strong>ica: HFES), pp. 365-369.<br />

Mäkelä. A., Fult<strong>on</strong>. And Suri, J., (2001), Supporting Users’ Creativity: Design to Induce Pleasurable Experiences.<br />

Proceedings of the Internati<strong>on</strong>al <str<strong>on</strong>g>C<strong>on</strong>ference</str<strong>on</strong>g> <strong>on</strong> affective human factors design, pp.387-394.<br />

149


Kalimullah Khan<br />

Norman, D. A. (2009), THE WAY I SEE IT - Memory is more important than actuality. Interacti<strong>on</strong>s Vol.16, No.2,<br />

pp. 24-26.<br />

Osgood, C. E., (1952), the nature and measurement of meaning. Psychological Bulletin, Vol. 49, No.3, pp. 197-<br />

237.<br />

Thurst<strong>on</strong>e, L.L., (1947), Multiple- Factor Analysis. Chicago: University of Chicago Press.<br />

UPA (Usability Professi<strong>on</strong>als’ Associati<strong>on</strong>). (2007), Usability body of knowledge. Internet Last Access: [2011-04-<br />

22] http://www.usabilitybok.org/glossary<br />

150


Challenges in Building a Community Health Informati<strong>on</strong><br />

Exchange in a Complex Envir<strong>on</strong>ment<br />

Ranjan Kini<br />

School of Business & Ec<strong>on</strong>omics, Indiana University Northwest, Gary, USA<br />

rkini@iun.edu<br />

Abstract: Ec<strong>on</strong>omists are projecting that the single most important cost to the world ec<strong>on</strong>omies in the future is<br />

the healthcare cost. Although more developed ec<strong>on</strong>omies are projecting the healthcare cost growth rate higher<br />

than their gross domestic product (GDP) growth rate, the projecti<strong>on</strong>s made for the US are dramatic enough to<br />

cause alarm and begin major nati<strong>on</strong>al c<strong>on</strong>versati<strong>on</strong>s. Unlike in Europe, the US healthcare has a larger share of<br />

its total healthcare costs privatized. Despite the fact that over half of US healthcare costs are borne by<br />

corporati<strong>on</strong>s and citizens, the government’s share, taking care of seniors, is projected to take a li<strong>on</strong>’s share of the<br />

GDP in the future and c<strong>on</strong>tinue to grow. In the US, this initiated studies of successful <str<strong>on</strong>g>European</str<strong>on</strong>g> and Asian<br />

nati<strong>on</strong>alized health care plans in identifying best practices. One of the first best practices that was pushed<br />

forward is to encourage establishment of nati<strong>on</strong>al-level, or state, or regi<strong>on</strong>al or community-level health<br />

informati<strong>on</strong> exchanges (HIE.) Adopting appropriate informati<strong>on</strong> and communicati<strong>on</strong> technology (ICT)<br />

infrastructure and creating such HIEs was identified as a strategy to address healthcare cost reducti<strong>on</strong>, quality<br />

gains, and safety in service provisi<strong>on</strong>. There are many HIEs that have been started in different regi<strong>on</strong>s of the US.<br />

Most of them, being early starters, were subsidized by federal agencies through grants. The objectives of<br />

supporting these exchanges were to identify the key ingredients necessary to develop and maintain sustainable<br />

HIEs. These reports from exchanges have identified many critical success factors that need to be addressed in<br />

building sustainable HIEs. However, these factors are influenced by the ICT infrastructure, community,<br />

competitive, and stakeholder characteristics of the regi<strong>on</strong>. C<strong>on</strong>sequently, in many parts of the US, the adopti<strong>on</strong> of<br />

HIE and subscribing to it has been slow. In this study, the challenges facing the establishment of a HIE in a<br />

regi<strong>on</strong> with three major competitive healthcare entities comprising of nine hospitals are studied. The research<br />

methodology that is used in this research to collect data is through semi-structured interviews with executives of<br />

three major healthcare providers (hospital groups). Although there are many stakeholders such as physicians,<br />

laboratories, pharmacies, etc., in making a HIE highly successful, this research is primarily focused <strong>on</strong> the central<br />

players of HIE, the hospitals. It is clear from the interview resp<strong>on</strong>ses that they are skeptical of any regi<strong>on</strong>al HIE<br />

becoming sustainable, and they believe that even for most HIEs, sustainability in the l<strong>on</strong>g run is difficult unless it<br />

is made mandatory and cost of doing business for stakeholders to c<strong>on</strong>trol the total healthcare costs. But, they do<br />

agree that there are opportunities for innovative HIEs to add business services to make them viable and<br />

sustainable.<br />

Keywords: health informati<strong>on</strong> exchange; regi<strong>on</strong>al health informati<strong>on</strong> exchange; sustainability of health<br />

informati<strong>on</strong> exchange; HIE and quality of health; health informati<strong>on</strong> technology<br />

1. Introducti<strong>on</strong><br />

It was in 2010, as I was getting admitted to the hospital for a surgery that I realized the dire need for<br />

transformati<strong>on</strong> in health informati<strong>on</strong> technology (HIT). Within a span of 3 hours, three different people<br />

came up to my bed and asked me the same set of questi<strong>on</strong>s and were writing <strong>on</strong> blank forms! During<br />

the third time as I was answering the questi<strong>on</strong>s, I asked the nurse the need for answering the same<br />

questi<strong>on</strong>s again and again. The answer was obvious that the systems in different departments wanted<br />

to collect and enter into their systems, separately! Again, in 2011, when a routine mammogram of a<br />

family member was needed to be reinterpreted for a sec<strong>on</strong>d opini<strong>on</strong>, the lab had to burn a CD to be<br />

taken to another radiologist. I realized at that time that we had a l<strong>on</strong>g way to go in transforming<br />

healthcare in the US through informati<strong>on</strong> and communicati<strong>on</strong> technologies (ICT). These types of<br />

situati<strong>on</strong>s are very comm<strong>on</strong> in the USA even in 2012. This is <strong>on</strong>e of the key reas<strong>on</strong>s federal<br />

government started incentivizing the use of ICT in healthcare.<br />

In the US, there is no doubt that the skills of physicians, and the quality of clinical technologies are of<br />

the highest level. But, according to a number of studies, the use of informati<strong>on</strong> technology in<br />

managing data both internally and externally am<strong>on</strong>g the healthcare stakeholders such as hospitals,<br />

physicians, pharmacies, labs, outpatient clinics, and payers is inefficient and can save up to 7% of the<br />

healthcare spending, i.e. ~100 billi<strong>on</strong> dollars, while increasing the quality of healthcare. C<strong>on</strong>sequently,<br />

federal government, which accounts for over 40% of the healthcare spending through Medicare and<br />

Medicaid, is incentivizing the use of ICT by all healthcare stakeholders. To encourage the use of ICT<br />

federal government’s first approach involved handing out competitive grants to regi<strong>on</strong>s, states and<br />

stakeholders with expectati<strong>on</strong> of l<strong>on</strong>g term sustainability of their projects. As the government<br />

witnessed the lack of initiatives <strong>on</strong> the part of many stakeholders, especially am<strong>on</strong>g physicians, it<br />

151


Ranjan Kini<br />

came out with an approach of demanding stakeholders to use ICT and file reports electr<strong>on</strong>ically to get<br />

reimbursement for the services they provided (through Medicare and Medicaid). In fact, by 2014, all<br />

physicians are expected to file reports which are c<strong>on</strong>sidered – “meaningful use of informati<strong>on</strong><br />

technology” –electr<strong>on</strong>ically, if not, Medicare is not expected to reimburse for their services. This<br />

approach has created some degree of panic and anxiety to affect structural changes in the healthcare<br />

industry.<br />

Another area federal government has identified is that cost reducti<strong>on</strong> and value can be gained is by<br />

having standardized health record. The healthcare industry identified that it should have three types of<br />

records: Electr<strong>on</strong>ic Medical Record, Electr<strong>on</strong>ic Health Record, and Electr<strong>on</strong>ic Patient Record.<br />

EMR definiti<strong>on</strong>: The data in the EMR -- clinical data, clinical decisi<strong>on</strong> support, c<strong>on</strong>trolled<br />

medical vocabulary, charge entry, computerized provider order entry, pharmacy<br />

informati<strong>on</strong>, drug interacti<strong>on</strong>s, and clinical documentati<strong>on</strong> applicati<strong>on</strong>s -- is the legal<br />

record of what happened to the patient during their encounter at the CDO and is owned<br />

by the CDO. (Ericksen, 2009)<br />

EHR definiti<strong>on</strong>: A subset of each care delivery organizati<strong>on</strong>’s (medical offices) presently<br />

assumed to be summaries of the patients C<strong>on</strong>tinuity of Care Record (CCR) or HL7’s<br />

C<strong>on</strong>tinuity of Care Document (CCD) which both can be simply called the patients<br />

electr<strong>on</strong>ic health record (EHR or EPR or PHR). This record is owned by the patient and<br />

has patient input and access that spans episodes of care across multiple CDOs within a<br />

community, regi<strong>on</strong>, or state. (Ericksen, 2009)<br />

The expectati<strong>on</strong> is that by having standardized EMR and EHR the health data of a healthcare<br />

encounter of a c<strong>on</strong>sumer with any of the stakeholders can easily be interchanged with other<br />

stakeholders who might use them as needed. The EMR which is yet to be standardized is typically<br />

created by each of the stakeholder using their proprietary vendor software; whereas an EHR is a<br />

record created by using summary of variety of EMRs from different stakeholders. The EHR is<br />

available to all stakeholders based <strong>on</strong> the needs of the service provider. But, the EHR is yet to be<br />

standardized and the federal government using the Nati<strong>on</strong>al Health Informati<strong>on</strong> Network (NHIN) as<br />

the base architecture encourages each state to define their EHR standard and establish an EHR<br />

repository for Health Informati<strong>on</strong> Exchange (HIE). In order to encourage different states and regi<strong>on</strong>s<br />

to start such HIEs Department of Health and Human Services (DHHS) initiated grant programs to<br />

form cooperatives, alliances, not-for-profit organizati<strong>on</strong>s as HIEs. Since 2005 there were 150 HIEs<br />

that were started but few (West, 2012) have become very successful and most of them are facing<br />

challenges to be sustainable. Although DHHS suggests forming state-level HIEs, the pi<strong>on</strong>eer of<br />

successful IHIE (Indiana Health Informati<strong>on</strong> Exchange), Mark Overhage (Health, 2006; Yasnoff, 2004)<br />

suggests that the regi<strong>on</strong>al HIEs makes more sense than state-level HIE. Regardless of whether a HIE<br />

is regi<strong>on</strong>al or state-level, business model that will be appropriate and sustainable for the HIE is the<br />

key interest of this study.<br />

2. Literature<br />

There have been many studies c<strong>on</strong>ducted evaluating and discussing the values and benefits of NHIN,<br />

HIE, and possible cost savings for both the federal government (Medicare & Medicaid) and payers<br />

(employers and insurance companies). Most of them promote the formati<strong>on</strong> of a network of HIEs.<br />

In 2001, Manl et al., suggested a standard EMR by all stakeholders, that is easily portable to any<br />

system, and also, available to patient through the web (securely) for periodic review, like pers<strong>on</strong>al<br />

bank account informati<strong>on</strong>, credit rating, etc. So far, however that has remained a dream.<br />

Yassenoff (2004) suggested, based <strong>on</strong> their study results, that for minimizing medical errors and<br />

enhancing quality of care, ICT is critical. The study pointed out that just by using computerized<br />

physician order entry (CPOE) in <strong>on</strong>e case reduced charge in <strong>on</strong>e instituti<strong>on</strong> by 12.7% and costs by<br />

13.1% and decreased serious medicati<strong>on</strong> errors by 55%. They estimated just by using CPOE<br />

nati<strong>on</strong>ally we can save 44 billi<strong>on</strong> dollars. They c<strong>on</strong>clude that the quality and efficiency of healthcare in<br />

the US can be improved just be using CPOE, readily accessible patient informati<strong>on</strong> and medical<br />

knowledge to support the physician at the time of need. They suggest a network of community of<br />

HIEs to support the associated local participating physicians with patient informati<strong>on</strong> for quality<br />

outcomes with savings.<br />

152


Ranjan Kini<br />

Many early studies pointed out that hospitals and other stakeholders were slow in adopting ICT to<br />

achieve higher stage of EMR adopti<strong>on</strong> model. (Health, 2006; West, 2012; Grossman, 2008) Some<br />

stakeholders pointed out that there was no <strong>on</strong>e standard agreed up<strong>on</strong> by all the stakeholders in terms<br />

of EMR. By 2013, providers are expected to migrate to ICD-10 Internati<strong>on</strong>al codes for EMR (Hans<strong>on</strong>,<br />

2012). As many instituti<strong>on</strong>s agreed and accepted Health Level 7 (HL7-CCD) as the good EHR<br />

standard, ASTM suggested C<strong>on</strong>tinuity Care Record (CCR) as a better EHR standard which sent both<br />

vendors as well as stakeholders back to the drawing board and are playing wait and see approach.<br />

This further slowed the adopti<strong>on</strong> of EMR and in turn EHR. Despite several successes of HIEs across<br />

the US many, with different models of HIEs, are still not c<strong>on</strong>vinced that EMR and EHR have been<br />

standardized or that quality of care has improved. (Greenemeier, 2009; EMR, 2011)<br />

In order to incentivize the start of Regi<strong>on</strong>al Health Informati<strong>on</strong> Organizati<strong>on</strong>s (RHIO) Federal<br />

government started supporting the initiatives through grant programs. Over 100+ such organizati<strong>on</strong>s<br />

were started, but, over a period of time, many of them became unsustainable (West, 2012). A few that<br />

have sustained have found it difficult to make a very str<strong>on</strong>g case for stakeholders to maintain a viable<br />

model.<br />

In Estahling (2005), Top Ten (2007), and Alfreds (2008) the authors emphasize that reas<strong>on</strong>s most of<br />

these RHIO or HIEs have not become successful is because of <strong>on</strong>e or more of the following factors<br />

were not properly addressed:<br />

Business Model: Clear understanding of the business model for HIE – how do we<br />

generate revenues since cost “savings” may cut revenue for some stakeholders.<br />

Collaborati<strong>on</strong> and Trust: Collaborative envir<strong>on</strong>ment and trust is not properly addressed<br />

by all HIE partners.<br />

Participati<strong>on</strong> role and benefits: The community and stakeholders needed to have clear<br />

and open discussi<strong>on</strong> about their c<strong>on</strong>tributi<strong>on</strong> to and gains from HIE.<br />

HIE Governance: The governance structure and processes need to be clearly<br />

understood by payers, providers, c<strong>on</strong>sumers, and all other stakeholders.<br />

Roadmap for HIE: A clear roadmap needs to be established for the community to<br />

understand and stakeholders to engage in the development.<br />

Value Propositi<strong>on</strong>: The value propositi<strong>on</strong>s of HIE are to be clearly articulated and<br />

communicated to all stakeholder groups.<br />

Communicati<strong>on</strong> of Returns: Financial costs and benefits, and Return <strong>on</strong> Investment of<br />

participati<strong>on</strong> to partners are to be clearly explained.<br />

Technology c<strong>on</strong>straints: Lack of a comm<strong>on</strong> HIE architecture/framework; lack of comm<strong>on</strong><br />

definiti<strong>on</strong>s and standards; lack of comm<strong>on</strong> standards for data creati<strong>on</strong> and exchange.<br />

Security and privacy risk: Perceived difficulty in addressing HIPAA requirements and<br />

data security is to be addressed.<br />

IT adopti<strong>on</strong>: Many stakeholders had limited their adopti<strong>on</strong> of HIT which in turn would<br />

delay their participati<strong>on</strong>.<br />

Incremental Movement: The community’s reluctance to move in the directi<strong>on</strong> of HIE<br />

initiative, creating difficulty in educating c<strong>on</strong>sumers and getting them to participate as<br />

stakeholders.<br />

Market Readiness and Awareness: Most stakeholders including payers and providers<br />

were not incentivized to build and/or participate in and HIE.<br />

The some or most of the above factors have been identified by several studies (Grossman, 2008) as<br />

the leading causes of poor sustainability of HIEs, nati<strong>on</strong>ally. The few currently sustaining models are<br />

c<strong>on</strong>stantly reenergizing their models. (West, 2012) For example, Indiana HIE is going across the<br />

border to Illinois to recruit providers, physicians and other stakeholders to increase their (subscripti<strong>on</strong><br />

and transacti<strong>on</strong>) revenue base. The l<strong>on</strong>g term sustainability of surviving HIEs need to find a viable<br />

model to generate revenue while working towards its objective of reducing the overall costs of<br />

healthcare and gain efficiencies.<br />

153


Ranjan Kini<br />

Typically, a HIE even though needs presence from all stake holders as shown in Figure 1, the<br />

possibility of them coming together to form a cooperative, or joint venture by leading partners, is<br />

c<strong>on</strong>sidered to be very difficult because of the factors menti<strong>on</strong>ed above. (Health, 2006)<br />

Figure 1: Health informati<strong>on</strong> exchange<br />

In Indiana, although the statewide IHIE has a presence in the Northwest Indiana regi<strong>on</strong>, authors,<br />

following the suggesti<strong>on</strong>s made in the several studies and reports (Health, 2006; Grossman, 2008;<br />

West, 2012) were interested in investigating the feasibility of an local HIE in the regi<strong>on</strong>.<br />

In Northwest Indiana, there are three hospital groups. First group has four hospitals (1500 beds) in<br />

the regi<strong>on</strong> but is a part of 14 hospital group in Indiana; sec<strong>on</strong>d group has two hospitals (600 beds) in<br />

the regi<strong>on</strong>; and third group has three hospitals (800 beds) in the regi<strong>on</strong>. These hospitals serve a<br />

regi<strong>on</strong> about half milli<strong>on</strong> populati<strong>on</strong>. The hospitals are competing for the revenue am<strong>on</strong>g themselves<br />

as well as many neighboring metropolitan hospitals of greater Chicago. Many of the physicians in the<br />

regi<strong>on</strong> are associated with and practice in competing hospitals. In additi<strong>on</strong> to these hospitals, there<br />

are several small hospitals of 40-beds or 50-beds in the regi<strong>on</strong>. Also, there are a variety of local<br />

laboratories and imaging centers as well as branches of nati<strong>on</strong>al laboratories. There are several<br />

nati<strong>on</strong>al and regi<strong>on</strong>al pharmacies competing with hospital pharmacies. Thus, potentially there are<br />

many possible stakeholders who are in a positi<strong>on</strong> to become part of a regi<strong>on</strong>al HIE, if made possible<br />

and feasible.<br />

The authors interest in the study is to understand the perspectives of these hospitals’ senior<br />

management regarding a HIE and to investigate their views <strong>on</strong> a Northwest Regi<strong>on</strong>al HIE. Thus,<br />

authors interviewed the hospital senior administrati<strong>on</strong> since in most HIEs hospitals are the anchor<br />

tenants and are the primary supporters needed to sustain a HIE. Most of the literature (listed<br />

references) relating to HIT and HIE has been studied to generate appropriate questi<strong>on</strong>s to interview<br />

the executives. These questi<strong>on</strong>s are based <strong>on</strong> the reported results and perspectives about a HIE from<br />

the point of view of other healthcare executives regarding the successes and failures of HIEs in the<br />

United States.<br />

Although earlier studies encourage starting a HIE, more specifically a regi<strong>on</strong>al HIE, they also warn<br />

that making a HIE sustaining and successful is a complex task. So, this study is c<strong>on</strong>ducted to<br />

154


Ranjan Kini<br />

investigate the perspectives that influence the decisi<strong>on</strong>-making in the formulati<strong>on</strong> of such a<br />

collaborative HIE. (Figure 1)<br />

3. Research methodology<br />

The literature <strong>on</strong> HIE indicate that perspectives of senior executives determine the formulati<strong>on</strong>,<br />

governance, success, and sustenance of a HIE. (Establishing, 2005; Health, 2006; Grossman, 2008;<br />

Alfreds, 2008) C<strong>on</strong>sequently, a set of questi<strong>on</strong>s were designed to develop it into a semi-structured<br />

interview questi<strong>on</strong>naire. (Health, 2006; Kvale, 1996) The expectati<strong>on</strong> is to get candid answers from<br />

senior executives of three hospital groups, separately. The interview was designed to be semistructured<br />

to allow room for follow-up questi<strong>on</strong>s if the questi<strong>on</strong>s are not clear or if there is a need to<br />

follow-up to clarify their comments. Furthermore, when resp<strong>on</strong>ders digressed into a topic of their<br />

interest, there were follow-up questi<strong>on</strong>s to bring the focus back in to HIE discussi<strong>on</strong>.<br />

Authors requested each hospital group’s main office to suggest an executive for the interview. The<br />

title of the executive of each of the hospital groups are: Manager of Clinical Informatics; Chief<br />

Financial Officer; Chief Financial Officer & Chief Informati<strong>on</strong> Officer; is also included in the Appendix<br />

1. Also, the edited resp<strong>on</strong>ses from the three interviews are included in the Appendix 1. (If the<br />

Appendix 1 is not included in the main paper for lack of space, up<strong>on</strong> request, it is available.)<br />

4. Results<br />

The first interview was with the Manager of Clinical Informatics of the hospital group of 14 hospitals,<br />

with four hospitals in the regi<strong>on</strong>. The manager resp<strong>on</strong>ded that they have been members of IHIE for<br />

some time. They pay their subscripti<strong>on</strong> as well as their physician associates fees, to access IHIE data.<br />

They pay for physicians’ IHIE subscripti<strong>on</strong> fee for fear of physician not admitting patients into their<br />

hospitals for lack of access to data, and go to competiti<strong>on</strong> which may provide that service. This<br />

executive does not think there is evidence at this time that quality of care has improved because of<br />

IHIE. But manager did acknowledge that IHIE’s presence in Illinois did help them retain the patients<br />

because 25 percent of their patients come from Illinois. This manager did not have an opini<strong>on</strong> about<br />

whether local, state or regi<strong>on</strong>al HIE will be more helpful, <strong>on</strong>e way or other. They did not think there<br />

were a large number of patients from other part of the state to make IHIE much more valuable. The<br />

manager did say they enter limited amount of data from their Epic (PHR) to IHIE’s (EHR). The<br />

manager thinks at this point there is no standard for EHR, thus each hospital decides to share and<br />

populate whatever data they see fit. This manager does not think there is going to be sustainable<br />

model for HIE and that their investment as a subscriber to IHIE is more a c<strong>on</strong>venience to physicians<br />

and protecti<strong>on</strong> from losing their business for not having it.<br />

The sec<strong>on</strong>d interview was with Vice President of Finance of a hospital group with two hospitals. This<br />

executive clearly is not a str<strong>on</strong>g supporter of HIE. They are not subscribers to HIE. They think<br />

eventually it may be mandated up<strong>on</strong> them until that time, they believer, they will be able to support<br />

their physicians through proprietary Epic system. The Epic system is used by all major hospitals in the<br />

regi<strong>on</strong>. Thus, as a result of c<strong>on</strong>tractual agreement with Epic, the physicians practicing in hospitals<br />

where Epic is used will have access to the patient data even if they are treating the patient in another<br />

hospital which also uses Epic. This executive believes that there is no large number of patients from<br />

outside the regi<strong>on</strong> to warrant subscripti<strong>on</strong> to IHIE. This is also the argument put forward by this<br />

executive <strong>on</strong> behalf of other stakeholders such as other small hospitals, labs, pharmacies etc., about<br />

subscribing to IHIE. This executive believes that bel<strong>on</strong>ging to IHIE is a charity strategy. The executive<br />

believes that cost is a sunk cost when organizati<strong>on</strong>s subscribe to HIE, and the <strong>on</strong>ly stakeholders that<br />

benefit from it are physicians and insurance companies. This executive supported their physicians to<br />

access Epic data from their private offices, by installing equipment and training them to use the<br />

system using federal grant m<strong>on</strong>ey. The executive does not anticipate any health care quality benefits<br />

from HIE at this time.<br />

The third group of hospitals was represented by both the chief financial officer and the chief<br />

informati<strong>on</strong> officer. These executives believed that HIEs are beneficial even if there is a high cost for<br />

subscripti<strong>on</strong>. They have subscribed to IHIE for the same reas<strong>on</strong> other (the first) hospital group did, to<br />

support the physicians to access the data (EHR) from anywhere. They also believe that membership<br />

in IHIE is beneficial since the report physicians need can be easily created by them from the data<br />

hospital populates in the IHIE data repository of EHR, thus saving work for hospital ICT department.<br />

They believe although there is no clear standard for EMR and EHR at this time, they think it will<br />

eventually happen. They believed that it does not really matter whether it is regi<strong>on</strong>al, state, or nati<strong>on</strong>al<br />

155


Ranjan Kini<br />

level HIE, it will work the same way and they are not very c<strong>on</strong>cerned about it. They also believes that<br />

the regi<strong>on</strong>al HIE in Northwest Indiana will have difficulty in sustaining itself in the l<strong>on</strong>g run, unless the<br />

HIE provides additi<strong>on</strong>al services to generate revenue. The executives also believed, like their<br />

competitors did, that Epic software does provide HIE type of service, through its proprietary PHR to<br />

physicians in the regi<strong>on</strong>; although executives think eventually HIE will be mandated by the federal<br />

government. These executives find the value in HIE but think there needs to be significant amount of<br />

work d<strong>on</strong>e to standardize the EMR, EHR, and middleware for interfacing with variety of existing<br />

software used by various stakeholders.<br />

The resp<strong>on</strong>ses given by the executives indicated that they are all c<strong>on</strong>cerned with the business model<br />

of HIEs. They believe that HIEs clearly have a role but the value of a HIE in terms of cost efficiencies<br />

gained and c<strong>on</strong>venience to physicians, and ease of patient data entry and data access is as<br />

expected. What they are really are not sure about are the advantages of quality of health delivered<br />

and revenue gain by <strong>on</strong>e or more stakeholders. They believe whether all the stakeholders are<br />

embracing HIE or not, so<strong>on</strong>er or later the federal government (the dominant payer) would require<br />

every stakeholders to c<strong>on</strong>nect into a HIE at regi<strong>on</strong>al, state or nati<strong>on</strong>al level.<br />

In this study, two of the three groups have already subscribed to the state level IHIE. They have also<br />

subsidized the access for the physicians both at the hospital as well as at physicians’ offices, so that<br />

there is minimal disc<strong>on</strong>tent am<strong>on</strong>g physicians. The group has indicated that although it has not<br />

subscribed to IHIE, its clinical system Epic is useful in accessing the Epic data (PHR – an Epic<br />

provisi<strong>on</strong> for clients) from neighboring hospitals as both other groups are using Epic. This hospital<br />

group has no immediate plans to become a subscriber of IHIE. According to the senior executive the<br />

loss of value will be felt <strong>on</strong>ly when a patient from a hospital without Epic presence is admitted in to<br />

their hospital. Thus, if Epic or similar clinical systems are in use at all or most of the hospitals in the<br />

regi<strong>on</strong>, and if the percentage of patients from outside the regi<strong>on</strong> admitted to these hospitals is very<br />

small then there may be opportunities for Epic type of product, with appropriate market penetrati<strong>on</strong> to<br />

be the default value provider rather than a HIE. Extending this logic, ignoring EMR and EHR<br />

standards for the moment, it is possible for a clinical system vendor to be the de facto HIE value<br />

provider. And, recently, vendors have been promoting this value to their clients to boost their<br />

marketshare.<br />

5. Discussi<strong>on</strong><br />

People need health care because: they fall sick as they get old; they get in to accidents during their<br />

normal living or activities; they are genetically pr<strong>on</strong>e to certain types of ailments because of their<br />

genetics; their envir<strong>on</strong>mental or living c<strong>on</strong>diti<strong>on</strong>s may cause certain types of ailments; they develop<br />

hobbies which are causes of certain types injuries or accidents; and, people develop vices which are<br />

to the detriment of their health. Of the list made above, except for the first three, which are not<br />

c<strong>on</strong>trollable by individuals, all the others are c<strong>on</strong>trollable by individuals. However, it is difficult to<br />

differentiate causes of certain healthcare costs as to which of the above caused the individual to<br />

encumber the medical expense. Payers try to manage the healthcare expenses while providing the<br />

best care, as defined in the c<strong>on</strong>tract, to individual subscribers while the cost of most healthcare<br />

products and services grow at a rate higher than all sectors of industry. Thus, it is logical that largest<br />

healthcare providers, the U.S. government (Medicare and Medicaid) and large employers would like<br />

to rein in the cost of healthcare. In this process <strong>on</strong>e of the first items that is brought to the attenti<strong>on</strong> of<br />

all stakeholders in the industry is reducing the cost of managing healthcare informati<strong>on</strong>.<br />

Although healthcare industry made tremendous progress in the use of computer technologies in<br />

clinical area its use in the informati<strong>on</strong> management lagged compared to all other industries. The<br />

management of informati<strong>on</strong> in the healthcare delivery has the potential to gain significant efficiencies<br />

and lower the cost of healthcare delivery but the adopti<strong>on</strong> and diffusi<strong>on</strong> of the use of ICT in the<br />

hospitals has been very slow. Thus, the largest payer, the federal government had to intervene and<br />

suggest the use of ICT by all stakeholders. The cost efficiencies gained through reduced paperwork,<br />

decreased fraud, enhanced quality of healthcare delivery, quality healthcare data for analysis and<br />

research are expected to be some of the major benefits from such interacti<strong>on</strong>s. NHIN needs to be in<br />

place to derive all these benefits. There are many ways of designing a NHIT and make it relevant to<br />

all citizens and stakeholders. A distributed network of HIE is <strong>on</strong>e such alternative. The number, the<br />

size, ownership, governance, sustainability all immediately became challenges of a HIE.<br />

156


Ranjan Kini<br />

The sustainability issue drives the HIE to find ways for stakeholders to cooperate, collaborate, govern<br />

within a highly competitive revenue declining industry. If providers and stakeholders were to decrease<br />

their costs equivalent to the subscripti<strong>on</strong> costs to HIE (or investment and governance in the formati<strong>on</strong><br />

of a HIE) then there is at least business case for an HIE. The IHIE seems to have found a way to<br />

make itself reas<strong>on</strong>ably sustainable through various value added activities such as Quality Health First<br />

(QHF) with “pay-for-value” as the motivating factor for linking physicians and payers/employers to<br />

become part of IHIE, and Health Informati<strong>on</strong> Technology for Ec<strong>on</strong>omic and Clinical Health (HITECH)<br />

to engage patients for public health and cost c<strong>on</strong>tainment. However, some providers have indicated<br />

that IHIE is expensive and are not sure whether they are generating enough value to make it<br />

worthwhile, especially when IHIE charges for custom c<strong>on</strong>solidated data analytic reports using their<br />

data repository over and above their subscripti<strong>on</strong> fees. Regardless, IHIE has developed a viable<br />

business model to influence at least two of this study hospital groups to not support the idea of a<br />

regi<strong>on</strong>al HIE in the Northwest Indiana but c<strong>on</strong>tinue with IHIE.<br />

If regi<strong>on</strong>al HIE were to be a viable business model, in the study regi<strong>on</strong> or elsewhere, then probably<br />

the investment made in to the ICT infrastructure may need to be properly leveraged. There is an<br />

opportunity for HIE to leverage their infrastructure to provide many stakeholders their data center<br />

needs for both clinical and financial data. This has a potential for significant savings for most small to<br />

medium size stakeholders: be it be physicians/groups; labs, clinics, imaging centers surgery centers<br />

etc. Independent HIEs can use the infrastructure to develop educati<strong>on</strong>al and knowledge support like<br />

in WebMD and also generate revenue sources through advertisements. But, this would demand<br />

aggressive innovative approaches from stakeholders or entrepreneurs!<br />

Acr<strong>on</strong>yms<br />

ACO – Accountable Care Organizati<strong>on</strong><br />

ASTM – American Society for Testing and Materials<br />

CCR - C<strong>on</strong>tinuity of Care Record or CCD – C<strong>on</strong>tinuity of Care Document<br />

CDO – Care Delivery Organizati<strong>on</strong>s<br />

CDR - Clinical Data Repository<br />

CDSS - Clinical Decisi<strong>on</strong> Support Systems<br />

CMV – C<strong>on</strong>trolled Medical Vocabulary<br />

CPOE - Computerized Provider Order Entry<br />

CRS - Care Record Summary<br />

EDI – Electr<strong>on</strong>ic Data Interchange<br />

EHR – Electr<strong>on</strong>ic Health Record<br />

EMR – Electr<strong>on</strong>ic Medical Record<br />

EPR – Electr<strong>on</strong>ic Patient Record<br />

Epic – Clinical System and Vendor name<br />

HIPPA- Health Insurance Portability and Accountability Act<br />

HL7 – Health Level 7 Internati<strong>on</strong>al Health Informati<strong>on</strong> Technology standards<br />

NHIN – Nati<strong>on</strong>al Health Informati<strong>on</strong> Network<br />

PHR – Patient Health Record<br />

RHIO - Regi<strong>on</strong>al Healthcare Informati<strong>on</strong> Organizati<strong>on</strong>s<br />

References<br />

Alfreds S. T., Masters E. T., & Himmelstein J. (2008) “Opportunities for Facilitating Electr<strong>on</strong>ic Health Informati<strong>on</strong><br />

Exchange in Publicly Funded Programs: Findings from Key Informant Interviews with Public Health Agency<br />

Leadership and Staff”, Center for Health Policy Research, University of Massachusetts, Medical School,<br />

Shrewsbury, MA, http://www.umassmed.edu/healthpolicy/HIT/PolicyDevelopment.aspx<br />

“EDI Today .HIE Tomorrow,” (2008), INGENIX, MN,<br />

http://www.optuminsight.com/c<strong>on</strong>tent/attachments/IX_PYR_CL_22409_EDItoday_WP.pdf<br />

“EMR Companies Holding Practice Data for ‘Ransom’” (2011) http://www.emrandhipaa.com/emr-andhipaa/2011/01/19/emr-companies-holding-practice-data-for-ransom/<br />

Eriksen, Andrew. (2009) “EMR vs. EHR Difference”, http://freeemrsoluti<strong>on</strong>.com/emr-articles/emr-vs-ehrdifference/<br />

“Establishing a Business Case for Health Informati<strong>on</strong> Exchange”, (2005) Findings from the State and Regi<strong>on</strong>al<br />

Dem<strong>on</strong>strati<strong>on</strong>s in Health Informati<strong>on</strong> Technology Regi<strong>on</strong>al Meeting, November 8 – 9,<br />

http://healthit.ahrq.gov/portal/server.pt/community/ahrq_nati<strong>on</strong>al_resource_center_for_health_it/650<br />

Garets, Dave & Davis, Mike (2006) “Electr<strong>on</strong>ic Medical Records vs. Electr<strong>on</strong>ic Health Records: Yes, There Is a<br />

Difference”, A HIMSS AnalyticsTM White Paper, HIMSS Analytics, Chicago, IL,<br />

http://www.himssanalytics.org.<br />

157


Ranjan Kini<br />

Greenemeier, Larry. (2009), “Will Electr<strong>on</strong>ic Medical Records Improve Health Care?”<br />

http://www.scientificamerican.com/article.cfm?id=electr<strong>on</strong>ic-health-records<br />

Grossman, J. M., Kushner, K. L., & November, E. A. (2008) “Creating Sustainable Local Health Informati<strong>on</strong><br />

Exchanges: Can Barriers to Stakeholder Participati<strong>on</strong> be Overcome?” Center for Studying Health<br />

System Change, NIHCM Foundati<strong>on</strong> Research Brief No. 2.<br />

Hans<strong>on</strong>, Wayne. (2012) “Will Affordable Care Act Uncertainties Affect Local IT?”<br />

http://www.digitalcommunities.com/articles/Will-Affordable-Care-Act-Uncertainties-Affect-Local-IT.html.<br />

“Health Informati<strong>on</strong> Exchange Projects: What Hospitals and Health Systems Need to Know, An<br />

Executive Brief,” (2006), by Manatt Health Soluti<strong>on</strong>s for American Hospital Associati<strong>on</strong>, Chicago,<br />

IL.<br />

Kvale, Steinar. (1996) Interviews: An Introducti<strong>on</strong> to Qualitative Research Interviewing, Sage Publicati<strong>on</strong>s,<br />

Thousand Oaks California.<br />

Mandl, K. D., Szolovits, P., Kohane, I.S. (2001) “Public standards and patients' c<strong>on</strong>trol: how to keep electr<strong>on</strong>ic<br />

medical records accessible but private,” British Medical Journal, (BMJ.com), Vol. 322, pp. 283-285.<br />

“Medical Records Privacy,” (2011) Published <strong>on</strong> Privacy Rights Clearinghouse,<br />

http://www.privacyrights.org, Revised January 2011.<br />

Mercuri John J. (2010) “The Ethics of Electr<strong>on</strong>ic Health Records,” http://www.clinicalcorrelati<strong>on</strong>s.org/?p=2211<br />

January 15, 2010<br />

Oracle Health Informati<strong>on</strong> Exchange: Secure, Seamless Data Sharing, (2010), Oracle Corporati<strong>on</strong>,<br />

http://www.oracle.com/us/industries/healthcare.<br />

Shay, Edward F. (2005) “Legal barriers to electr<strong>on</strong>ic health records,”<br />

http://www.physiciansnews.com/law/505.html<br />

Stafford, Nancy. (2010) “Who owns the data in an Electr<strong>on</strong>ic Health<br />

Record?”http://www.ehrinstitute.org/articles.lib/items/who-owns-the-data-in<br />

“Top Ten Success Factors for Community HIE,” (2007) White Paper, Center for Community Health Leadership,<br />

Raleigh, NC, 2007.<br />

West, D. M. & Friedman, A. (2012) “Health Informati<strong>on</strong> Exchanges and Megachange,” Governance Studies,<br />

Brookings Instituti<strong>on</strong>, Washingt<strong>on</strong>, DC.<br />

Yasnoff, W. A., Humphreys, B. L., Overhage, M., Detmer, D. E., Brennan, P. F., Morris, R. W., Middlet<strong>on</strong>, B.,<br />

Bates, D. W., Fanning, J. P. (2004) “A C<strong>on</strong>sensus Acti<strong>on</strong> Agenda for Achieving the Nati<strong>on</strong>al Health<br />

Informati<strong>on</strong> Infrastructure,” Journal of the American Medical Informatics Associati<strong>on</strong>, Vol. 11 No. 4, 2004,<br />

pp. 332-338.<br />

158


Factors Inhibiting Recogniti<strong>on</strong> and Reporting of Losses<br />

From Cyber-Attacks: The Case of Government<br />

Departments in the Western Cape Province of South Africa<br />

Michael Kyobe 1 , Sinka Matengu 1 , Proske Walter 1 and Mzwandile Sh<strong>on</strong>gwe 2<br />

1<br />

Department of Informati<strong>on</strong> Systems, University of Cape Town, Cape Town,<br />

South Africa<br />

2<br />

Department of Informati<strong>on</strong> Studies, University of Zululand, Kwadlangezwa,<br />

South Africa<br />

michael.kyobe@uct.ac.za<br />

msh<strong>on</strong>gwe@pan.uzulu.ac.za<br />

Abstract: The South African government has invested substantially in IT to improve service delivery and benefit<br />

from the low cost of communicati<strong>on</strong> via e.g., the internet. However, cybercrime, lack of accountability and failure<br />

to evaluate e-developments remain a major c<strong>on</strong>cern to the society. The level of awareness of these risks in this<br />

sector appears to be low despite effort to address these challenges in government and private sector initiatives<br />

and c<strong>on</strong>ferences. The draft paper <strong>on</strong> informati<strong>on</strong> security (Dept of Public service and administrati<strong>on</strong> n.d. p2) and<br />

the CSIRT initiative clearly point to the fact that success in government electr<strong>on</strong>ic initiatives depends <strong>on</strong> effective<br />

informati<strong>on</strong> security management. The present study examined some of the factors inhibiting the recogniti<strong>on</strong> and<br />

reporting of losses from cyber-attacks <strong>on</strong> government departments in South Africa. A survey was c<strong>on</strong>ducted in<br />

the Western Cape Province. Forty resp<strong>on</strong>ses were received and analysed using mixed methods. The results<br />

indicate that lack of clear guidance <strong>on</strong> how to calculate losses; lack of understanding of the legislati<strong>on</strong> and<br />

knowledge of how it may assist in cubing cyber-crime; lack of training and creati<strong>on</strong> of awareness of cyber-crime<br />

and lack of knowledge and capability to assess risks regularly are major factors inhibiting departments from<br />

recognising and reporting losses from cyber attacks.<br />

Keywords: cybercrime, human behaviours, IT, informati<strong>on</strong> security, public sector, South Africa<br />

1. Introducti<strong>on</strong><br />

The government of South Africa, like other governments around the world, has invested in modern<br />

Informati<strong>on</strong> and Communicati<strong>on</strong> Technologies (ICTs) to improve service delivery in the public sector<br />

(Syväjärvi et al. 2009; Nyanda 2010). C<strong>on</strong>sequently, reliance <strong>on</strong> informati<strong>on</strong> systems (IS) has<br />

increased and placed government operati<strong>on</strong>s at higher security risk (News24 2007). However, the<br />

level of awareness of security risks in the public sector appears to be low and recogniti<strong>on</strong> and<br />

reporting of losses resulting from cyber attacks is a major challenge (Nyanda 2010; De Tolly, Maumbe<br />

and Alexander 2006).<br />

With the increasing attacks <strong>on</strong> government systems (Kayle 2010), it is critical that such attacks are<br />

effectively detected, identified and mitigated and early warning to potential targets raised. There is<br />

growing c<strong>on</strong>cern however that limited resources are devoted to informati<strong>on</strong> security and accounting<br />

for the actual cost of losses. Informati<strong>on</strong> security challenges may not be addressed with limited<br />

knowledge of the actual costs of breaches (Cashell, Jacks<strong>on</strong>, Jickling and Webel 2004). In many<br />

jurisdicti<strong>on</strong>s, for cyber attacks to be c<strong>on</strong>sidered statutory crime they must exceed a specified amount.<br />

Insurance firms will also require loss estimati<strong>on</strong>s in order to determine damages and recoveries. This<br />

therefore means that accurate record keeping and proper evaluati<strong>on</strong> of cyber losses are necessary.<br />

Recent studies show however that poor record-keeping systems c<strong>on</strong>tinue to be a major barrier to<br />

instituti<strong>on</strong>al, legal and regulatory reform; anti-corrupti<strong>on</strong> strategies; poverty reducti<strong>on</strong> and ec<strong>on</strong>omic<br />

development (De Tolly, Maumbe and Alexander 2006; Mari<strong>on</strong> 2008). The purpose of this research is<br />

to investigate factors that impede South African government departments in the Western Cape<br />

Province, from recognising and reporting losses due to cybercrime. In the first secti<strong>on</strong>, we present the<br />

literature review <strong>on</strong> cybercrime and the theoretical work explaining the difficulties involved in<br />

recognising and reporting losses from cyber attacks. The informati<strong>on</strong> security challenges in the public<br />

sector are then discussed and some measures taken to mitigate these challenges are presented. A<br />

research model is developed and the methodology for the present study. Finally the research findings<br />

are discussed and recommendati<strong>on</strong>s are made.<br />

159


2. Literature review<br />

Michael Kyobe et al.<br />

While no agreed up<strong>on</strong> definiti<strong>on</strong> exists for the term cybercrime, it can generally be thought of as any<br />

criminal activity involving a computer system (Kshetri 2009). Today the world experiences large scale<br />

ever evolving attacks, which are blended, malicious in nature and involving thousands of computers.<br />

The process of identifying, recognising and reporting losses from cybercrime depends <strong>on</strong> many<br />

factors including our understanding of what cybercrime represents, methods used in risk identificati<strong>on</strong><br />

and analysis, the design of systems and human attitude or behaviour (Canhoto 2010). Li<strong>on</strong>el and V<strong>on</strong><br />

Frederick (2005) argue that crime involves many things and is committed in many different forms by<br />

different agents. C<strong>on</strong>sequently it would be unlikely that the scope and power of its facets can be<br />

captured by a single theory (Li<strong>on</strong>el & V<strong>on</strong> Frederick 2005). In the following secti<strong>on</strong>s, we discuss some<br />

theoretical perspectives of electr<strong>on</strong>ic crime and the challenges involved in its recogniti<strong>on</strong> and<br />

reporting.<br />

2.1 Lack of understanding of cybercrime impact its recogniti<strong>on</strong><br />

In his positi<strong>on</strong> paper to the Oxford Internet Institute, Baker (2010) argues that the process of collecti<strong>on</strong><br />

and analysis of cybercrime data is often affected by a lack of understanding of what cybercrime<br />

means or represents. This lack of understanding of cybercrime, which is also evidenced by various<br />

ambiguous and c<strong>on</strong>flicting interpretati<strong>on</strong>s of the term, impedes its recogniti<strong>on</strong> and measurement.<br />

Canhoto (2010) also observed that this lack of understanding is complicated by the fact that the types<br />

of crimes organisati<strong>on</strong>s m<strong>on</strong>itor are different and as such comm<strong>on</strong> strategies in addressing crime<br />

challenges may be problematic. For instance, crime m<strong>on</strong>itored by financial instituti<strong>on</strong>s may not be<br />

relevant for law enforcement agencies. She maintains that the process of detecting cybercrime must<br />

be c<strong>on</strong>ducted in a correct way and that the process of interpreting the data objects is c<strong>on</strong>diti<strong>on</strong>ed by<br />

the many factors including the positi<strong>on</strong> in the organisati<strong>on</strong>, the characteristics of the envir<strong>on</strong>ment and<br />

the cognitive processes.<br />

2.2 Informati<strong>on</strong> asymmetries<br />

Informati<strong>on</strong> asymmetry deals with the study of decisi<strong>on</strong>s in transacti<strong>on</strong>s where <strong>on</strong>e party has more or<br />

better informati<strong>on</strong> than the other. This creates an imbalance of power in transacti<strong>on</strong>s such that the<br />

ignorant party will lack negotiating capabilities or lack ability to retaliate in event of a breach of<br />

agreement.<br />

Asymmetric informati<strong>on</strong> can be a str<strong>on</strong>g impediment to effective security. For instance, many<br />

instituti<strong>on</strong>s collect data about security breaches and fail to share it with their stakeholders. Banks for<br />

instance do not want to share informati<strong>on</strong> <strong>on</strong> fraud and cyber-attacks resulting in shortage of relevant<br />

informati<strong>on</strong> to fight crime effectively. It has also been reported that some organisati<strong>on</strong>s inflate<br />

reported crime statistics (Levi et al. 2007) while others like Internet Service Providers may understate<br />

crime committed by their customers.<br />

Christensen and Laegreid (2007) discuss several bureaucratic, political and hierarchical structures<br />

and policies that make cross-boundary sharing of informati<strong>on</strong> and decisi<strong>on</strong> making about priorities,<br />

resources and systems difficult in the public sector. For instance, some privacy acts restrict the<br />

sharing of some kind of data between government instituti<strong>on</strong>s and business partners. Shortage of<br />

hard data about informati<strong>on</strong> security failures therefore becomes a major impediment to determining<br />

losses and c<strong>on</strong>taining crime.<br />

2.3 Fragmentati<strong>on</strong> of legislati<strong>on</strong> and law enforcement<br />

The fragmentati<strong>on</strong> of jurisdicti<strong>on</strong>s hinders rapid resp<strong>on</strong>se. A crime committed by some<strong>on</strong>e in a<br />

different country may be difficult to investigate due to differences or lack of appropriate legislati<strong>on</strong> and<br />

bi-lateral agreements. Phishermen often send hot m<strong>on</strong>ey through the banks of member States with a<br />

relaxed attitude to asset recovery resulting in m<strong>on</strong>ey laundering. Also traditi<strong>on</strong>al mechanisms for<br />

internati<strong>on</strong>al police cooperati<strong>on</strong> have been found to be too slow and expensive to cope with the<br />

demands of the Internet age.<br />

2.4 Other ec<strong>on</strong>omic factors<br />

In his presentati<strong>on</strong> <strong>on</strong> how ec<strong>on</strong>omics and informati<strong>on</strong> security affect cybercrime, Guerra (2009)<br />

outlines several important ec<strong>on</strong>omic issues. For instance, that crime can be extremely profitable, has<br />

160


Michael Kyobe et al.<br />

low overhead and a little risk of being prosecuted. Guerra adds further that business and government<br />

have less m<strong>on</strong>ey to spend <strong>on</strong> security even as the threat grows. Similar c<strong>on</strong>cerns have been<br />

expressed by other writers. Anders<strong>on</strong> and Moore (2006) discuss the problem of misaligned incentives,<br />

whereby the pers<strong>on</strong> who is resp<strong>on</strong>sible for securing a system or reporting security breaches has no<br />

incentive to do so as it does not affect them (or do not stand to benefit from ensuring security).<br />

2.5 Informati<strong>on</strong> systems security challenges in the public sector<br />

2.5.1 Cybercrime in the public sector in South Africa<br />

Managing cyber-crime in the 46 public departments is a challenging task. A draft policy <strong>on</strong> cyber<br />

security was <strong>on</strong>ly gazetted in 2009 by the then minister of Communicati<strong>on</strong> Siphiwe Nyanda. According<br />

to this draft, South Africa does not have a coordinated approach to dealing with cybercrime and while<br />

various structures were in place to deal with cyber security, these were perceived to be inadequate in<br />

dealing with cyber security (Sapa 2010; V<strong>on</strong> Solms 2012).<br />

Limited empirical studies <strong>on</strong> cyber crime in South Africa have been c<strong>on</strong>ducted and as such there is<br />

much reliance <strong>on</strong> news articles and <strong>on</strong>line reports (Kayle 2011). However, organised crime syndicates<br />

are increasingly targeting government systems. According to the Informati<strong>on</strong> Security Group (ISG), it<br />

is estimated that the department of Home Affairs lost around R400 milli<strong>on</strong> in 2009. In a case of fraud<br />

in January 2009, an employee of CIPRO colluded with syndicates to divert R51 milli<strong>on</strong> of tax refunds<br />

from South African Revenue Services - SARS (Kayle 2010). In September 2008, a court sentenced<br />

two SARS employees and their accomplices to 15 years impris<strong>on</strong>ment after each had defrauded the<br />

taxman of nearly R500 000 (Otto, 2008).<br />

Other African countries have experienced some level of cyber-crime. There are a few studies though<br />

that report <strong>on</strong> cyber-crime in Africa. Wangwe, Eloff and Venter (2009) reported that cyber-fraud<br />

remains a big c<strong>on</strong>cern in East Africa. Heeks (2002) reports that in many African countries, data quality<br />

and data security are very poor and there are few mechanisms to address these issues. He states<br />

further that digital signatures cannot be accepted in some countries. He c<strong>on</strong>cludes that while there<br />

have been some progress in e-government, much has to be accomplished in terms of computing and<br />

telecommunicati<strong>on</strong>s infrastructure in Africa. These sentiments are also shared by Gichoya (2009) in<br />

Kenya. Adomi and Igun (2007) reported that Nigeria is another African country that is experiencing<br />

cybercrime. They state that, this has led to Nigerian email accounts and networks being blocked in<br />

other countries. This shows that Africa is also not safe from cyber criminals.<br />

2.5.2 Lack of accounting, legal and IT skills<br />

Some of the problems relating to the escalati<strong>on</strong> of cybercrime and poor security in Government<br />

departments could be attributed to the lack of skills in accounting, law and IT (The Auditor General<br />

2010). According to Mari<strong>on</strong> (2008), the Auditor General found that municipalities are failing to<br />

account for funds meant for service delivery because of a critical shortage of staff and lack of internal<br />

c<strong>on</strong>trols. Of the 138 municipalities whose books were audited in 2008, <strong>on</strong>ly two received clean audits.<br />

Fifty municipalities received adverse and disclaimer opini<strong>on</strong>s, meaning that either their financial<br />

statements were ‘fundamentally flawed’ or informati<strong>on</strong> submitted could not be corroborated with any<br />

documentati<strong>on</strong> respectively. Nombembe said most of these qualificati<strong>on</strong>s were mainly due to a lack of<br />

adequate internal c<strong>on</strong>trols; lack of discipline to retain supporting documents and lack of capacity. In<br />

KwaZulu-Natal, according to the Business Day, out of 54 audited municipalities, <strong>on</strong>ly 17 received<br />

unqualified audit opini<strong>on</strong>s. There were three municipalities that improved from a qualified to an<br />

unqualified opini<strong>on</strong> and the number of disclaimers decreased. However, the number of worst-case<br />

opini<strong>on</strong>s remained high at 28%, as two municipalities received adverse opini<strong>on</strong>s and 13 received<br />

disclaimer opini<strong>on</strong>s.”<br />

Another problem relate to the unwillingness by managers to report cyber attack incidents. In many<br />

cases the victims withhold reporting (Kyobe 2005). The importance of reporting incidents has been<br />

emphasised in many studies <strong>on</strong> informati<strong>on</strong> security and safety. G<strong>on</strong>zalez (2005) maintains that<br />

informati<strong>on</strong> security reporting is a quality improvement process that is essential to reduce incidents.<br />

EURIM (2003) outlines several barriers to reporting including c<strong>on</strong>cern about c<strong>on</strong>fidentiality, disrupti<strong>on</strong><br />

to business and loss of reputati<strong>on</strong>. Lack of m<strong>on</strong>itoring and evaluati<strong>on</strong> and inappropriate budget for<br />

informati<strong>on</strong> security have also been identified.<br />

161


Michael Kyobe et al.<br />

3. Initiatives to address these challenges<br />

However, there have been several measures taken by the government and researchers to address<br />

cybercrime challenges. Recently, the Department of Communicati<strong>on</strong>s published the draft Cyber<br />

security Policy of South Africa. The Electr<strong>on</strong>ic Communicati<strong>on</strong> and Transacti<strong>on</strong> (ECT) Act (2002) <strong>on</strong>e<br />

effective law put in place by the government to govern the use of informati<strong>on</strong> technology in South<br />

Africa. The problems relating to cybercrime are addressed by the cybercrime secti<strong>on</strong> in Chapter XIII of<br />

the ECT Act, 2002. According to Michals<strong>on</strong> and Hughes (2005), this chapter introduces statutory<br />

criminal offenses relating to unauthorized access to data (e.g., through hacking), intercepti<strong>on</strong> of data<br />

(e.g., tapping into data flows or denial of service attacks), interference with data (e.g., viruses) and<br />

computer related extorti<strong>on</strong>s, fraud and forgery. They also state that a pers<strong>on</strong> aiding those involved in<br />

these crimes will be guilty as an accessory. A pers<strong>on</strong> c<strong>on</strong>victed of an offence related to the above is<br />

liable to a fine or impris<strong>on</strong>ment for a period not exceeding five years. There has also been effort<br />

toward the establishment of a Computer Security Incident Resp<strong>on</strong>se Team (CIRT) and c<strong>on</strong>ferences<br />

have been organised to address informati<strong>on</strong> security. CSIRT has however been affect by funding and<br />

implementati<strong>on</strong> challenges.<br />

4. C<strong>on</strong>ceptual model<br />

The above review indicates many factors influence recogniti<strong>on</strong> and reporting of losses from cyberattacks<br />

<strong>on</strong> government departments. Those examined in the present study are: human behaviour<br />

(i.e., attitude of managers toward informati<strong>on</strong> security) (Dowland et al. 1999); lack of understanding of<br />

the regulati<strong>on</strong>s governing the use of electr<strong>on</strong>ic media (Dowland et al. 1999); and lack of accounting<br />

skills and methods required to prepare losses (Bougaardt and Kyobe 2011). Figure 1 represents the<br />

c<strong>on</strong>ceptual model and the relati<strong>on</strong>ships between the elements of the model. We propose that human<br />

behaviour (e.g. attitude of top management and staff members towards Informati<strong>on</strong> Security);<br />

understanding of the IT regulati<strong>on</strong> that deals with cybercrime; and possessi<strong>on</strong> of relevant accounting<br />

skills and methods influence the ability to recognise and prepare loss estimates arising from<br />

cybercrime in government departments. This propositi<strong>on</strong> is tested and the results are presented in the<br />

following secti<strong>on</strong>s.<br />

Human Behaviors<br />

Possessi<strong>on</strong> of<br />

Accounting skills &<br />

methods<br />

Understanding of IT<br />

regulati<strong>on</strong>s<br />

Ability to recognize<br />

and report losses<br />

from cyber attacks<br />

Figure 1: Factors influencing recogniti<strong>on</strong> and reporting of losses from cyber-attacks <strong>on</strong> government<br />

departments<br />

5. Research methodology<br />

This study was c<strong>on</strong>ducted using a questi<strong>on</strong>naire. This was c<strong>on</strong>sidered to be appropriate since<br />

resp<strong>on</strong>dents were located in different areas. The questi<strong>on</strong>naire was made available either as an<br />

<strong>on</strong>line versi<strong>on</strong> or a paper-based hard copy for the c<strong>on</strong>venience of the resp<strong>on</strong>dents. The <strong>on</strong>line versi<strong>on</strong><br />

was created and hosted <strong>on</strong> Google Documents. The questi<strong>on</strong>naire c<strong>on</strong>sisted of three secti<strong>on</strong>s:<br />

Secti<strong>on</strong> 1 c<strong>on</strong>tained questi<strong>on</strong>s <strong>on</strong> cyber-attacks and its recogniti<strong>on</strong>. A Likert scale with opti<strong>on</strong>s 1<br />

(str<strong>on</strong>gly disagree) to 5 (str<strong>on</strong>gly agree) was used. In secti<strong>on</strong> 2 resp<strong>on</strong>dents were asked to give<br />

additi<strong>on</strong>al informati<strong>on</strong> that they felt was relevant to the study. Secti<strong>on</strong> 3 captured the demographic<br />

informati<strong>on</strong> about the resp<strong>on</strong>dents. The questi<strong>on</strong>naire was piloted with <strong>on</strong>e academic and commerce<br />

students and a few correcti<strong>on</strong>s were made.<br />

162


6. Data sample<br />

Michael Kyobe et al.<br />

The sample populati<strong>on</strong> was made up of employees and managers at various public organisati<strong>on</strong>s<br />

which have adopted ICT in the provisi<strong>on</strong> of services to the public. A list of public organisati<strong>on</strong>s located<br />

in Cape Town was obtained from the government website www.gov.za. Stratified Random Sampling<br />

technique was used to select 28 organisati<strong>on</strong>s offering different services and invitati<strong>on</strong>s were then<br />

sent out requesting for their participati<strong>on</strong> in the study. 20 departments in Transport Services,<br />

Telecommunicati<strong>on</strong>, Arts and Culture, Metro Police, Development Corporati<strong>on</strong>, Nati<strong>on</strong>al Library, City<br />

of Cape Town, Premier’s Office, South Africa Reserve Bank (SARB) and the Auditor General’s office<br />

indicated their willingness to participate. All these organisati<strong>on</strong>s use ICT in their operati<strong>on</strong>s.<br />

We focused <strong>on</strong> public instituti<strong>on</strong>s in Cape Town due to their close proximity and to allow for easier<br />

follow-up <strong>on</strong> the resp<strong>on</strong>ses. The survey was sent to each of the organizati<strong>on</strong>s and this had to be<br />

completed by IT managers or officers, accounting officers, and general managers. IT managers were<br />

best equipped to answer the majority of the questi<strong>on</strong>s and also to give an understanding of the<br />

technology and processes in place at the organisati<strong>on</strong>. Accounting managers were deemed to provide<br />

the detail <strong>on</strong> procedures and methods used to record any losses as a result of cybercrimes, while<br />

general managers could provide general informati<strong>on</strong> relating to the organisati<strong>on</strong>.<br />

120 questi<strong>on</strong>naires were sent out in total and <strong>on</strong>ly 40 usable questi<strong>on</strong>naires were returned. This<br />

provided a very low resp<strong>on</strong>se rate of about 30%. Some of the reas<strong>on</strong>s for poor resp<strong>on</strong>ses were that<br />

some public officers did not have permissi<strong>on</strong> from the executive heads to participate in surveys.<br />

However, as indicated in the profiles of the resp<strong>on</strong>ded below, most of these resp<strong>on</strong>dents hold senior<br />

positi<strong>on</strong>s in the organisati<strong>on</strong>s, and possess expertise in the fields of IT, Accounting and <strong>Management</strong>.<br />

Hence the informati<strong>on</strong> obtained from this sample is valuable.<br />

7. Data analysis and results<br />

Data was analysed using quantitative techniques and the results are presented in the following<br />

secti<strong>on</strong>s. Resp<strong>on</strong>dents c<strong>on</strong>sisted of Admin/General Manager, IT Managers, HR Managers,<br />

Accounting, Audit and Risk <strong>Management</strong>, Directors, IT Support including those in Health services,<br />

Librarian, IT Developers, and others. The sample c<strong>on</strong>sisted of a fair representati<strong>on</strong> of different IT<br />

users in the government departments. The results are presented in table 1.<br />

Table 1: Professi<strong>on</strong> of the resp<strong>on</strong>dents<br />

Professi<strong>on</strong> No. of Resp<strong>on</strong>dents<br />

Admin General Manager 6<br />

IT Managers 10<br />

HR Managers 5<br />

Accounting, Audit and Risk <strong>Management</strong> 5<br />

Directors 2<br />

IT Support 3<br />

Librarian 1<br />

IT developers 4<br />

Others 4<br />

Table 2 below presents the mean resp<strong>on</strong>ses of these participants. On average, the resp<strong>on</strong>dents did<br />

not possess appropriate accounting skills which suggest lack of skills <strong>on</strong> how to prepare and account<br />

for losses arising from cyber-attacks. In additi<strong>on</strong>, many indicated unawareness and unhappiness with<br />

the methods or procedures followed in determining the costs of such attacks. Many were also<br />

uncertain about the regulatory requirements and were not sure if the ECT Act provided sufficient<br />

remedy to cyber problems. They did not know if proper documentati<strong>on</strong> procedures <strong>on</strong> security were<br />

maintained and disagreed that their behaviours and those of managers facilitated the recogniti<strong>on</strong> and<br />

reporting of losses.<br />

Table 2: Mean resp<strong>on</strong>ses (Items measuring C<strong>on</strong>structs)<br />

C<strong>on</strong>struct No. of Mean Std. Dev.<br />

items<br />

Possessi<strong>on</strong> of accounting skills and methods 5 2.18 0.77<br />

Knowledge of Legislati<strong>on</strong> 5 3.23 0.82<br />

Human Behaviour 7 2.17 0.65<br />

Recogniti<strong>on</strong> and reporting of losses 4 2.72 0.78<br />

163


Michael Kyobe et al.<br />

We c<strong>on</strong>ducted the Cr<strong>on</strong>bach Alpha test to determine the reliability of the resp<strong>on</strong>ses. The results are<br />

presented in Table 3 below.<br />

Table 3: Results of the Cr<strong>on</strong>bach Alpha test<br />

C<strong>on</strong>struct Cr<strong>on</strong>bach’s Alpha<br />

Possessi<strong>on</strong> of Accounting skills & Methods 0.94<br />

Knowledge of Legislati<strong>on</strong> 0.78<br />

Human Behaviours 0.72<br />

Recogniti<strong>on</strong> and preparati<strong>on</strong> of losses 0.67<br />

Except for the c<strong>on</strong>struct (Recogniti<strong>on</strong> and Reporting of losses), the Alpha values for the other<br />

c<strong>on</strong>struct exceed the threshold of 0.70 thereby c<strong>on</strong>firming the reliability of their measures. The low<br />

reliability results obtained for Recogniti<strong>on</strong> and Reporting of losses may be attributed to the few items<br />

that measured this c<strong>on</strong>struct and also the fact that the sample size was small.<br />

Furthermore, factor analysis was c<strong>on</strong>ducted to determine the loading of each item <strong>on</strong> its respective<br />

c<strong>on</strong>struct. Table 4 shows those items that loaded 0.50 or more <strong>on</strong> the factors.<br />

Table 4: Factor analysis results<br />

C<strong>on</strong>struct Qn Factor 1<br />

(Accounting)<br />

Factor 2<br />

(Recogniti<strong>on</strong><br />

and Reporting)<br />

Factor 3<br />

(Human<br />

Behaviours)<br />

Factor 4<br />

(Regulati<strong>on</strong>)<br />

Possessi<strong>on</strong> of Accounting<br />

Skills<br />

17 0.512361 -0.112675 0.1674321 0,432861<br />

18 0.623493 0.121041 -0.132770 0.147611<br />

19 0.781132 0.054068 0.112183 0.006652<br />

20 0.613003 -0.070995 0.128150 -0.001822<br />

22 0.721991 0.256773 0.075049 0.047416<br />

Knowledge of Legislati<strong>on</strong> 9 0.092453 0.114719 0.244063 0.542167<br />

10 -0.000126 0.142647 -0.01478 0.544206<br />

11 0.401819 -0.372891 -0.008176 0.500121<br />

Human behaviours 1 -0.14410 0.557550 0.520341 -0.064458<br />

2 0.056432 -0.054231 0.553762 0.115432<br />

3 0.278451 0.342772 0.532280 -0.005477<br />

7 0.258186 0.298509 0.548337 -0.040771<br />

5 0.443221 0.100453 0.542765 0.136745<br />

15 0.315871 -0.120605 0.883331 0.250380<br />

Recogniti<strong>on</strong> & Reporting 4 0.116744 0.671662 0.012634 0.016934<br />

6 -0.177945 0.549261 0.013467 0.478221<br />

21 0.412734 0.671608 -0.253795 0.034560<br />

22 0.628032 0.604773 0.004249 0.122333<br />

* Items with factor loading of 0.50 or more are highlighted<br />

7.1 Discussi<strong>on</strong> of results<br />

Four factors were extracted as presented in Table 3 above. Items that loaded <strong>on</strong> Factor 1 were items<br />

Q17, Q18, Q19, Q20 and Q22. Q22 was however excluded because it loaded <strong>on</strong> two c<strong>on</strong>structs.<br />

Therefore Factor 1 represents the Accounting c<strong>on</strong>struct since most of the items measured this<br />

c<strong>on</strong>struct. Availability of clear guidance <strong>on</strong> how to calculate losses (Q19) had the highest loading <strong>on</strong><br />

this factor. Table 1 c<strong>on</strong>firms that resp<strong>on</strong>dents did not agree that they possessed adequate skills in<br />

accounting and accounting methods and that the method used in their organisati<strong>on</strong>s did not reflect the<br />

true costs of cyber-attacks. Further, resp<strong>on</strong>ses to Q18 also indicate that there were no clear<br />

procedures <strong>on</strong> how to report security breaches. It is therefore not surprising that few could report <strong>on</strong><br />

security breaches in their departments (Q17). Such lack of proper guidance and training in<br />

appropriate accounting techniques affect recogniti<strong>on</strong> of losses (Hood and Rothstein, 2000).<br />

Only three of the five items that measured resp<strong>on</strong>dents understanding of IT regulati<strong>on</strong>s loaded <strong>on</strong><br />

Factor 2 (i.e., Q9, Q10 and Q11). This factor represents the Regulati<strong>on</strong> c<strong>on</strong>struct. The percepti<strong>on</strong> of<br />

the level of support the ECT Act provides in cubing cybercrime loaded the highest (Q10), followed by<br />

resp<strong>on</strong>dent’s understanding of how the Act applied to security matters and existence of<br />

documentati<strong>on</strong> procedures to ensure compliance with the Act. Table 1 (Average resp<strong>on</strong>ses) indicates<br />

164


Michael Kyobe et al.<br />

that many were uncertain about the role of the ECT Act. Being uncertain about what the ECT Act can<br />

do might have also c<strong>on</strong>tributed to the lack of implementati<strong>on</strong> of measures to ensure compliance.<br />

Further analysis of Q9 (I understand how the ECT Act applies to security) shows that <strong>on</strong>ly IT<br />

managers understood well the requirements of the ECT Act. 72% of the resp<strong>on</strong>dents did not know the<br />

effectiveness of the ECT Act.<br />

The elements that loaded to the c<strong>on</strong>struct (Human behaviours) were Q1, Q3, Q7, Q5 and Q15 (My<br />

organisati<strong>on</strong> provides security and awareness training <strong>on</strong> security) loaded the highest. Table 1<br />

(Average results) suggests limited training d<strong>on</strong>e <strong>on</strong> risk management. This lack of training and<br />

understanding of risks means that crime is committed without the awareness of system users and as<br />

such can not be recognised and reported (Fick 2009). Resp<strong>on</strong>ses to Q3 and Q18 also indicate that<br />

resp<strong>on</strong>dents were not certain if they had a specific IT risk policy and that various methods were being<br />

employed to address the problem. Without a clear policy <strong>on</strong> security and risk management, managers<br />

are left to apply different strategies which could be inc<strong>on</strong>sistent and sometimes c<strong>on</strong>fusing.<br />

The lack of training may also be attributed to poor budget allocati<strong>on</strong> <strong>on</strong> security. According to<br />

Richards<strong>on</strong> (2008), inappropriate allocati<strong>on</strong> of security budget may result in limited funds or resources<br />

for security. It is not surprising therefore that Q1, and Q7, for instance, received negative resp<strong>on</strong>ses.<br />

Resp<strong>on</strong>ses to Q1 show that informati<strong>on</strong> security was perceived to be of least importance to top<br />

management by n<strong>on</strong>-managerial staff. In additi<strong>on</strong>, members did not know their obligati<strong>on</strong> towards<br />

informati<strong>on</strong> security therefore m<strong>on</strong>itoring and evaluati<strong>on</strong> of informati<strong>on</strong> security could not be enforced<br />

effectively by management (Q5). Q15 also had the highest loading overall – emphasising the<br />

importance of awareness and training <strong>on</strong> security.<br />

The last factor represents the c<strong>on</strong>struct Recogniti<strong>on</strong> and reporting of losses. Items Q4, Q6, and Q21<br />

loaded <strong>on</strong> it. Q4 (risks are regularly assessed in the organisati<strong>on</strong>) and Q21 (all breaches we<br />

experience are included in the costs incurred) loaded highly. The requirements for proper Informati<strong>on</strong><br />

security documentati<strong>on</strong> are not adhered to by the departments. Departments do not appear to keep<br />

appropriate security records. This is c<strong>on</strong>sistent with the observati<strong>on</strong>s of Mari<strong>on</strong> (2008). A culture of<br />

poor documentati<strong>on</strong>s may also discourage managers from preparing loss reports or report attack<br />

incidents (G<strong>on</strong>zalez 2005; Eurim 2003).<br />

8. C<strong>on</strong>clusi<strong>on</strong>, limitati<strong>on</strong>s and recommendati<strong>on</strong>s<br />

The present study examined some of the factors inhibiting the recogniti<strong>on</strong> and reporting of losses<br />

from cyber-attacks <strong>on</strong> government departments in the Western Cape Province of South Africa. The<br />

results indicate that lack of clear guidance <strong>on</strong> how to calculate losses; lack of understanding of the<br />

legislati<strong>on</strong> and knowledge of how it may assist in cubing cybercrime; lack of training and creati<strong>on</strong> of<br />

awareness of cybercrime and lack of knowledge and capability to assess risks regularly are major<br />

factors inhibiting departments from recognising and reporting losses from cyber attacks.<br />

Lack of proper guidance <strong>on</strong> determining losses means that accurate and reliable data for analysis<br />

cannot be available, nor can these departments develop necessary competencies in assessing risks.<br />

The lack of knowledge about the regulati<strong>on</strong> and its implicati<strong>on</strong>s may be attributed to the culture of not<br />

sharing informati<strong>on</strong> typical of public instituti<strong>on</strong>s. Training in the area of informati<strong>on</strong> security awareness<br />

is imperative, and appears to be a worldwide problem for the public sector. Research c<strong>on</strong>ducted by<br />

PriceWaterhouseCoopers (2012) shows that there has been degradati<strong>on</strong> in employee security<br />

awareness training; disaster recovery planning and informati<strong>on</strong> security management strategies in the<br />

public sector. The negative resp<strong>on</strong>ses received for the c<strong>on</strong>struct “Possessi<strong>on</strong> of Accounting skills and<br />

Methods” causes much c<strong>on</strong>cern. It imply that very few public sector departments have adequate<br />

accounting methods to allow for correct estimati<strong>on</strong> of cybercrime related incidents. These findings are<br />

c<strong>on</strong>sistent with internati<strong>on</strong>al finding by Smith (2004) and Richards<strong>on</strong> (2008). A recent study by the HP<br />

and Ec<strong>on</strong>omist Intelligent Unit also c<strong>on</strong>firms that public sector organizati<strong>on</strong>s world-wide have been<br />

historically slow to share informati<strong>on</strong> between and within organizati<strong>on</strong>s (Rodgers 2010).<br />

We str<strong>on</strong>gly recommend that government departments and other stakeholders engage in initiatives to<br />

create awareness of the cyber-risks. While the government c<strong>on</strong>tinues to invest in technology and<br />

related mechanisms for detecti<strong>on</strong> and preventi<strong>on</strong>, it is equally important to develop a culture of<br />

sharing informati<strong>on</strong> <strong>on</strong> cybercrime and development of skills in accounting and management of cyber<br />

risks.<br />

165


Limitati<strong>on</strong>s<br />

Michael Kyobe et al.<br />

When evaluating the results of this study it should be noted that the sample size was a limiting factor<br />

with 40 resp<strong>on</strong>ses being recorded. This may have impacted <strong>on</strong> the statistical analysis of the data.<br />

Another major setback is that we obtained very few comments from the resp<strong>on</strong>dents to verify most of<br />

the resp<strong>on</strong>ses obtained. In most cases many could not provide such comments because of lack of<br />

authority to report <strong>on</strong> certain sensitive issues. Therefore not much of the qualitative analysis could be<br />

d<strong>on</strong>e as initially planned. It should further be noted that resp<strong>on</strong>dents came exclusively from the<br />

Western Cape, possibly giving a skewed view of the public sector and an unrepresentative<br />

demographical makeup of resp<strong>on</strong>dents.<br />

Recommendati<strong>on</strong>s for future research<br />

The sample size should be increased by targeting other geographic locati<strong>on</strong>s to ensure correct<br />

demographics of the public sector. Interviews and open ended questi<strong>on</strong>s can provide further insight<br />

into the meaning of the data collected and analysed. It would also be interesting to determine the<br />

relati<strong>on</strong>ship between the variables and the strength of each inhibiting factor using statistical methods<br />

like regressi<strong>on</strong> analysis.<br />

References<br />

Acts <strong>on</strong>line (2002) Electr<strong>on</strong>ic Communicati<strong>on</strong> and Transacti<strong>on</strong> Act 2002, [Online] Available:<br />

http://www.acts.co.za/ect_act/index.htm [23 August 2010].<br />

Adomi, Esharenana E. and Igun, Stella E. (2008),’Combating cyber crime in Nigeria’, The Electr<strong>on</strong>ic Library, vol.<br />

26, no. 5, pp. 716 – 725.<br />

Anders<strong>on</strong>, R. and More, T. (2006) “The Ec<strong>on</strong>omics of Informati<strong>on</strong> Security”, Science Magazine, vol. 314, no.<br />

5799, pp. 610 – 613.<br />

Baker, W.H. (2010) Thoughts <strong>on</strong> Mapping and Measuring Cybercrime.<br />

Oxford Internet Institute Forum Mapping and Measuring Cybercrime, [Online]<br />

Available http://www.sfu.ca/~icrc/c<strong>on</strong>tent/oxford.forum.cybercrime.pdf [10 March 2011].<br />

Bougaardt, G and Kyobe, M. (2011) ‘Investigating the Factors Inhibiting SMEs From Recognizing and Measuring<br />

Losses From Cyber Crime in South Africa’. The Electr<strong>on</strong>ic Journal Informati<strong>on</strong> Systems Evaluati<strong>on</strong>, vol. 14,<br />

no. 2, pp. 67-178. [Online] Available: www.ejise.com [26 June 2012].<br />

Canhoto, A. (2010) What’ before ‘How’, Oxford Internet Institute Forum, [Online], Available:<br />

http://www.sfu.ca/~icrc/c<strong>on</strong>tent/oxford.forum.cybercrime.pdf [24 February 2011].<br />

Cashell, B., Jacks<strong>on</strong>, W.D., Jickling, M.and Webel, B. (2004) CRS Report for C<strong>on</strong>gress, The Ec<strong>on</strong>omic Impact of<br />

Cyber-Attacks, [Online] Available:<br />

http://www.cisco.com/warp/public/779/govtaffairs/images/CRS_Cyber_Attacks.pdf [13 March 2012]<br />

Christensen, T. and Lægreid, P. (2007b) The whole-of-government approach to public sector reform, PAR.<br />

Public Administrati<strong>on</strong> Review, vol. 67, no.6, pp. 1059-1066.<br />

Christensen, T. and Lægreid, P. (2007a). Introducti<strong>on</strong> - Theoretical Approach and Research Questi<strong>on</strong>s, in T.<br />

Christensen and P. Lægreid (eds.): Transcending New Public <strong>Management</strong>, Aldershot: Ashgate.<br />

De Tolly, K., Maumbe, B. and Alexander, H. (2006) Rethinking E-government development: Issues, Less<strong>on</strong>s and<br />

Future Prospects for the Cape Gateway Portal in South Africa, in Cunningham, Paul and Cunningham,<br />

Miriam (Eds): IST-Africa 2006 <str<strong>on</strong>g>C<strong>on</strong>ference</str<strong>on</strong>g> Proceedings, IIMC Internati<strong>on</strong>al Informati<strong>on</strong> <strong>Management</strong><br />

Corporati<strong>on</strong>, 2006, [Online] Available: http://www.ist-africa.org/<str<strong>on</strong>g>C<strong>on</strong>ference</str<strong>on</strong>g>2006/ [24 August 2010].<br />

Dept of Public service and administrati<strong>on</strong> (n.d) Draft positi<strong>on</strong> Paper <strong>on</strong> Informati<strong>on</strong> Security, [Online], Available:<br />

http://www.dpsa.gov.za/documents/acts&regulati<strong>on</strong>s/frameworks/ecommerce/POSITION%20PAPER%20ON%20INFORMATION%20SECURITY1.pdf.<br />

[24 August 2010].<br />

Dowland, P.S., Furnell, S.M., Illingworth, H.M. and Reynolds, P.L. (1999) “Computer crime and abuse: a survey<br />

of abuse and awareness”, Computers and Security, vol. 18, no. 8, pp. 715-726.<br />

EURIM (2003) IPPR E-Crime study, Partnership policing for the Informati<strong>on</strong> Society, Working Paper 1: Reporting<br />

methods and structures, [Online] Available: http://www.eurim.org/c<strong>on</strong>sult/ecrime/dec03/ECS_WP1_web_031209.pdf<br />

[24 November 2010].<br />

Gichoya, D. (2009). Facing the challenges of ICT implementati<strong>on</strong> in Government. IST-Africa 2009 c<strong>on</strong>ference<br />

proceedings, Paul Cunningham and Miriam Cunningham (Eds). IIMC, 2009.<br />

G<strong>on</strong>zalez, J.J. (2005) Towards a Cyber Security Reporting System – A Quality Improvement Process,<br />

<str<strong>on</strong>g>C<strong>on</strong>ference</str<strong>on</strong>g> Proceedings, 24th Internati<strong>on</strong>al <str<strong>on</strong>g>C<strong>on</strong>ference</str<strong>on</strong>g> <strong>on</strong> Computer Safety, Reliability, and Security,<br />

Fredrikstad, pp. 368–380.<br />

Guerra, P. (2009) How ec<strong>on</strong>omics and informati<strong>on</strong> security affects cybercrime and what this means in the c<strong>on</strong>text<br />

of a global recessi<strong>on</strong>, [Online] Available: http://www.blackhat.com/presentati<strong>on</strong>s/bh-usa-<br />

09/GUERRA/BHUSA09-Guerra-Ec<strong>on</strong>omicsCyberCrime-SLIDES.pdf [ 29 August 2010].<br />

Heeks R. (2002). eGovernment in Africa: Promise and Practice, Paper 13, Institute for Development Policy and<br />

<strong>Management</strong>. http://idpm.man.ac.uk/wp/igov/index.htm<br />

166


Michael Kyobe et al.<br />

Kayle, A. (2010) Experts Criticise Cyber Policy, [Online] Available:<br />

http://www.itweb.co.za/index.php?opti<strong>on</strong>=com_c<strong>on</strong>tent&view=article&id=31353:experts-criticise-cyberpolicy&catid=159:it-in-banking&tmpl=comp<strong>on</strong>ent&print=1<br />

[23 March 2010].<br />

Kayle, A. (2011) SA Lacks accurate cyber crime statistics, [Online] Available:<br />

http://www.itweb.co.za/index.php?opti<strong>on</strong>=com_c<strong>on</strong>tent&view=article&id=46586 [23 March 2010].<br />

Kshetri, N. (2009) “Positive Externality, Increasing Returns, and the Rise in Cybercrimes”, Communicati<strong>on</strong>s of the<br />

ACM, vol. 52, December, pp. 141-144.<br />

Kyobe, M.E. (2005) Addressing e-crime and computer security issues in small organisati<strong>on</strong>s in South Africa, EMT<br />

2005 Proccedings, The <str<strong>on</strong>g>European</str<strong>on</strong>g> <strong>Management</strong> and Technology <str<strong>on</strong>g>C<strong>on</strong>ference</str<strong>on</strong>g> <strong>on</strong> integrati<strong>on</strong> of management<br />

and technology, Rome, Italy, June 2005.<br />

Levi, M., Burrows, J. Fleming, M. and Hopkins, M. (2007) The nature, extent and ec<strong>on</strong>omic impact of fraud in the<br />

UK, L<strong>on</strong>d<strong>on</strong>: Associati<strong>on</strong> of Chief Police Officers. [Online] Available:<br />

http://www.cardiff.ac.uk/socsi/resources/ACPO%20final%20nature%20extent%20and%20ec<strong>on</strong>omic%20imp<br />

act%20of%20fraud.pdf [01 May 2011].<br />

Li<strong>on</strong>el C.M. and V<strong>on</strong>Frederick, R. (2005). Theories of Crime Causati<strong>on</strong>. [Online], Available:<br />

http://www.v<strong>on</strong>frederick.com/pubs/Theories%20of%20Crime%20Causati<strong>on</strong>.pdf. [ 25 August 2010].<br />

Mari<strong>on</strong>, C.C. (2008) Kwazulu Natal Anti Corrupti<strong>on</strong> Strategy, [Online] Available:<br />

http://www.kwazulunatal.gov.za/premier/anti_fraud/Presentati<strong>on</strong>_To_Kzn_Provincial_Anti_Corrupti<strong>on</strong>_Sum<br />

mit.pdf [20 August 2010].<br />

Michals<strong>on</strong>, L. and Hughes, B. (2005) Guide to the ECT Act, [Online] Available : http://www.michals<strong>on</strong>.com [20<br />

August, 2009].<br />

News24 (2007) Huge growth in cybercrime, [Online] Available: http://www.news24.com/SouthAfrica/News/Hugegrowth-in-cyber-crime-20071114<br />

[14 April 2010].<br />

Nyanda, S. (2010) Notice of Intenti<strong>on</strong> to Make South African Nati<strong>on</strong>al Security Cybersecurity Policy, Government<br />

Gazette Vol. 536. No 32963, [Online] Available: http://www.pmg.org.za/files/docs/100219cybersecurity.pdf<br />

[30 April 2010].<br />

Otto, H. (2008) SARS Employees get 15 years for fraud, [Online] Available:<br />

http://www.iol.co.za/index.php?set_id=1&click_id=15&art_id=vn20080905054713259C961747 [24 April<br />

2010].<br />

PricewaterhouseCoopers LLP (2012) Eye of the Storm: Key findings from 2012 Global State of Informati<strong>on</strong><br />

Security Survey, [Online] Available: http://www.pwc.se/sv_SE/se/riskhantering/assets/2012-global-state-ofinformati<strong>on</strong>-security-survey.pdf<br />

[13 March 2012].<br />

Rodgers, G. (2010) Managing informati<strong>on</strong> effectively: a necessity for the public sector, Hewlett-Packard<br />

Development Company (2010), Business White Paper. [Online] Available:<br />

http://www.managementthinking.eiu.com/sites/default/files/Government%20Workflow%20and%20Security_<br />

0.pdf [20 December 2011].<br />

Syväjärvi, A., Stenvall, J., Laitinen, I. and Harisalo, R. (2009) Informati<strong>on</strong> <strong>Management</strong> as Functi<strong>on</strong> of Data<br />

Mining and ICT In City Government, [Online] Available:<br />

http://www.egpa2009.com/documents/psg1/Anitti.pdf [27 August 2010].<br />

The Auditor General (2010) 2009-10 PFMA audit outcomes - Provincial c<strong>on</strong>solidated, [Online] Available<br />

http://www.pmg.org.za/files/docs/110208c<strong>on</strong>solidatedreport.pdf [20June 2011].<br />

V<strong>on</strong> Solms, B. (2012) Cyber crime challenges, [Online] Available: http://www.eepublishers.co.za/article/cybercrime-challenges.html<br />

[14 February 2012].<br />

Wangwe, Carina K., Eloff, Mariki M., and Venter, Lucas M. (2009). e-Government Readiness: An Informati<strong>on</strong><br />

Security Perspective from East Africa. IST-Africa 2009 c<strong>on</strong>ference proceedings, Paul Cunningham and<br />

Miriam Cunningham (Eds). IIMC, 2009.<br />

167


The Overall Process Taken by Enterprises to Manage the<br />

IaaS Cloud Services<br />

Alina Mădălina L<strong>on</strong>ea 1 , Daniela Elena Popescu 2 and Octavian Proştean 1<br />

1 Automati<strong>on</strong> and Applied Informatics Department, Faculty of Automati<strong>on</strong> and<br />

Computers, “Politehnica” University of Timisoara, Timisoara, Romania<br />

2 Computer Engineering Department, Faculty of Electrical Engineering and<br />

Informati<strong>on</strong> Technology, University of Oradea, Oradea, Romania<br />

madalina_l<strong>on</strong>ea@yahoo.com<br />

depopescu@uoradea.ro<br />

octavian.prostean@aut.upt.ro<br />

Abstract: Small and medium-sized enterprises (SMEs) were the initial focus for cloud services and they are<br />

susceptible to a c<strong>on</strong>tinuous adopti<strong>on</strong> of cloud computing services, because of its str<strong>on</strong>g advantages of accessing<br />

data from any place in the world over the Internet without c<strong>on</strong>cerning about the infrastructure used and the<br />

problems involved by the installati<strong>on</strong> and maintenance processes. However, organizati<strong>on</strong>s need to c<strong>on</strong>sider<br />

simultaneously both risks and rewards within the decisi<strong>on</strong> making process, in order to assure an efficient expertise.<br />

SMEs represent the target group of this study c<strong>on</strong>cerned with the outsourcing process to Cloud Service Provider<br />

(CSP) c<strong>on</strong>sidering the fact that the number of SMEs is greater than the number of large organizati<strong>on</strong>s, making<br />

SMEs the heart of ec<strong>on</strong>omies worldwide (Sharma, et al., 2010; Van Hoecke, et al., 2011). The aim of the proposed<br />

research represents a qualitative analysis of the overall process taken by SMEs to manage the migrati<strong>on</strong> of their<br />

applicati<strong>on</strong>s to Infrastructure-as-a-Service (IaaS). We c<strong>on</strong>ducted a literature analysis using papers released both<br />

by academic and practiti<strong>on</strong>er bodies, in order to resp<strong>on</strong>d to the following two research questi<strong>on</strong>s: What are the<br />

steps involved in the migrati<strong>on</strong> process of the SMEs to cloud services? What are the stages required by each step<br />

of the outsourcing process? In this sense we produced a theoretical process, which includes a collecti<strong>on</strong> of the<br />

following interrelated activities: data analysis step, decisi<strong>on</strong> making step, migrati<strong>on</strong> step and management step. In<br />

an IaaS cloud service, the CSP supports the hardware related issues, whilst the software related issues should be<br />

identified by enterprises that want to migrate to cloud. Thus, this paper is first proposing to address an overview of<br />

the data analysis step. This c<strong>on</strong>stitutes the initial step of the overall process taken by organizati<strong>on</strong>s and it<br />

comprises: the analysis of cloud migrati<strong>on</strong> opportunities, the study of cloud adopti<strong>on</strong> barriers and the examinati<strong>on</strong><br />

of current infrastructure used by the organizati<strong>on</strong>. Further, another objective of this paper is to address the<br />

decisi<strong>on</strong> making step, which implies the following decisi<strong>on</strong>s: what informati<strong>on</strong> should be moved into cloud and who<br />

will access the informati<strong>on</strong>, what CSP the organizati<strong>on</strong> will choose and how the organizati<strong>on</strong> will manage the cloud<br />

services. The decisi<strong>on</strong>s will be made based <strong>on</strong> the analysis step. We assumed that the cloud service type was<br />

chosen (i.e. IaaS) and the cloud deployment model was selected as well (i.e. public cloud). Furthermore, the<br />

effective moving stage of enterprise’s assets into cloud services is the migrati<strong>on</strong> step, which includes two<br />

activities: developing the Service Level Agreement (SLA) and implementing cloud. In additi<strong>on</strong>, the last step of the<br />

overall process is the management step, which is realized using two management functi<strong>on</strong>s: business and<br />

operati<strong>on</strong>al.<br />

Keywords: cloud management, outsourcing, IaaS, SME, cloud risks, cloud benefits, service level agreement<br />

1. Introducti<strong>on</strong><br />

Informati<strong>on</strong> Systems (IS) has a great impact for the business growth of Small and Medium sized<br />

Enterprises (SMEs) and it started with pers<strong>on</strong>al computers in order to manage the day-to-day<br />

operati<strong>on</strong>s of the enterprises using the basic applicati<strong>on</strong>s (i.e. word processing and accounting<br />

systems), complex applicati<strong>on</strong>s (i.e. decisi<strong>on</strong>al support systems) and the services produced in the<br />

Internet age (i.e. email, web sites, transacti<strong>on</strong> processing systems) (Levy and Powell, 2004). Today,<br />

enterprises adhere to cloud computing technology, which is subject to a c<strong>on</strong>tinuous development and<br />

it is c<strong>on</strong>sidered the future and the improvement of Informati<strong>on</strong> and Communicati<strong>on</strong> Technology (ICT).<br />

While in 2000 the tendency of Small and Medium sized Enterprises (SMEs) was to migrate to the<br />

Enterprise Resource Planning (ERP) soluti<strong>on</strong>s (Adam and O'Doherty, 2000), today ICT assists to a<br />

trend of SMEs to migrate from the traditi<strong>on</strong>al SMEs to the SMEs based cloud. The migrati<strong>on</strong> of<br />

enterprises to cloud is because of the advantages offered by this technology, defined by Joe<br />

Weinmann (2011) as an acr<strong>on</strong>ym: Comm<strong>on</strong>, Locati<strong>on</strong>-Independent Online Utility <strong>on</strong>-Demand service,<br />

<strong>on</strong> the Axiomatic Cloud Theory. However, simultaneously with the increased number of enterprises<br />

that adopt cloud computing, the challenges of enterprises to exploit cloud for their business objectives<br />

are growing as well. Thus, companies go through a holistic process in order to manage the<br />

implemented cloud services.<br />

168


Alina Mădălina L<strong>on</strong>ea, Daniela Elena Popescu and Octavian Proştean<br />

SMEs represent the target group of this study c<strong>on</strong>cerned with the outsourcing process to Cloud<br />

Service Provider (CSP) c<strong>on</strong>sidering the fact that the number of SMEs is greater than the number of<br />

large organizati<strong>on</strong>s, making SMEs the heart of ec<strong>on</strong>omies worldwide (Sharma, et al., 2010; Van<br />

Hoecke, et al., 2011). SME is a collective term for all micro, small or medium enterprises, which is<br />

qualified based <strong>on</strong> the following characteristics defined by the <str<strong>on</strong>g>European</str<strong>on</strong>g> Commissi<strong>on</strong><br />

Recommendati<strong>on</strong> (2003/361/EC): maximum number of staff (i.e. less than 250), annual turnover (e.g.<br />

≤ 50 milli<strong>on</strong> Euros) or an annual balance sheet (e.g. total ≤ 43 milli<strong>on</strong> Euros) (Begg and Caira, 2012).<br />

However, these characteristics do not influence the amount and nature of organisati<strong>on</strong>al data for<br />

SMEs. Despite the fact that SMEs are not sufficiently oriented toward the ICT soluti<strong>on</strong>s like the large<br />

organizati<strong>on</strong>s (Dai, 2009; Levy and Powell, 2004), SMEs started to explore the movement benefits to<br />

cloud services (Begg and Caira, 2012) for both business models (i.e. companies with existing IT<br />

infrastructure and start-up companies). Moreover, Devos, Van Landeghem and Deschoolmeester<br />

(2012) state that SME presents insufficient internal IT expertise and needs external IT expertise.<br />

Nevertheless, Misra and M<strong>on</strong>dal (2011) observed that the start-up SMEs area presents a str<strong>on</strong>ger<br />

interest for adopting cloud based services. Van Hoecke, et al. (2011) present their awareness<br />

regarding the migrati<strong>on</strong> of SMEs with existing IT infrastructure to hybrid cloud and the outsourcing of<br />

resources to public clouds for the start-up SMEs.<br />

Companies can choose from a wide range of cloud services (i.e. Infrastructure-as-a-Service, Platformas-a-Service<br />

and Software-as-a-Service), which can be deployed in four deployment models (i.e.<br />

private cloud, public cloud, hybrid cloud and community cloud). The selecti<strong>on</strong> of the cloud<br />

deployment model depends <strong>on</strong> the size of the organizati<strong>on</strong> and its Informati<strong>on</strong> Technology (IT)<br />

maturity level. While SMEs would rather prefer to outsource their applicati<strong>on</strong>s within an external cloud<br />

provider, the large organizati<strong>on</strong>s first take into c<strong>on</strong>siderati<strong>on</strong> the soluti<strong>on</strong> of having a private cloud and<br />

after that, they can decide to migrate their n<strong>on</strong>-critical informati<strong>on</strong> (i.e. test and development) to public<br />

deployments (CSCC, 2011). Additi<strong>on</strong>ally, depending <strong>on</strong> the IT maturity level of the enterprise, the<br />

SMEs will move from the traditi<strong>on</strong>al SMEs to the SMEs based cloud in order to gain access to a wide<br />

range of advanced IT applicati<strong>on</strong>s like: Enterprise Resource Planning (ERP), Customer Relati<strong>on</strong>ship<br />

<strong>Management</strong> (CRM), Human Resources (HR) and Collaborati<strong>on</strong> tools (KPMG, 2011). Sharma, et al.<br />

(2010) dem<strong>on</strong>strates the benefits of using Enterprise Resource Planning (ERP) based cloud services<br />

instead of traditi<strong>on</strong>al ERP system by SMEs.<br />

N<strong>on</strong>etheless, the aim of the proposed research represents a qualitative analysis of the overall process<br />

taken by SMEs to manage the migrati<strong>on</strong> of their applicati<strong>on</strong>s to Infrastructure-as-a-Service (IaaS).<br />

We c<strong>on</strong>ducted a literature analysis using papers released both by academic and practiti<strong>on</strong>er bodies,<br />

in order to resp<strong>on</strong>d to the following two research questi<strong>on</strong>s: What are the steps involved in the<br />

migrati<strong>on</strong> process of the SMEs to cloud services? What are the stages required by each step of the<br />

outsourcing process? In this sense we produced a theoretical process, which includes a collecti<strong>on</strong> of<br />

the following interrelated activities: data analysis step, decisi<strong>on</strong> making step, migrati<strong>on</strong> step and<br />

management step. In an IaaS cloud service, the CSP supports the hardware related issues, whilst the<br />

software related issues should be identified by enterprises that want to migrate to cloud.<br />

Thus, this paper is first proposing to address an overview of the data analysis step. This c<strong>on</strong>stitutes<br />

the initial step of the overall process taken by organizati<strong>on</strong>s and it comprises: the analysis of cloud<br />

migrati<strong>on</strong> opportunities, the study of cloud adopti<strong>on</strong> barriers and the examinati<strong>on</strong> of current<br />

infrastructure used by the organizati<strong>on</strong>. Further, another objective of this paper is to address the<br />

decisi<strong>on</strong> making step, which implies the following decisi<strong>on</strong>s: what informati<strong>on</strong> should be moved into<br />

cloud and who will access the informati<strong>on</strong>, what CSP the organizati<strong>on</strong> will choose and how the<br />

organizati<strong>on</strong> will manage the cloud services. Furthermore, the effective moving stage of enterprise’s<br />

assets into cloud services is the migrati<strong>on</strong> step, which includes two activities: developing the Service<br />

Level Agreement (SLA) and implementing cloud. In additi<strong>on</strong>, the last step of the overall process is the<br />

management step, which is realized using two management functi<strong>on</strong>s: business and operati<strong>on</strong>al.<br />

The remainder of this paper is structured as follows: secti<strong>on</strong> 2 discusses the proposed management<br />

process of enterprise’s migrati<strong>on</strong> to IaaS; secti<strong>on</strong> 3 reviews the related work in this area; while secti<strong>on</strong><br />

4 c<strong>on</strong>cludes the paper.<br />

2. The management process of enterprise’s migrati<strong>on</strong> to IaaS<br />

From c<strong>on</strong>sumer perspectives, the overall process taken by enterprises to manage the IaaS cloud<br />

services includes a collecti<strong>on</strong> of the following interrelated activities: data analysis step, decisi<strong>on</strong><br />

169


Alina Mădălina L<strong>on</strong>ea, Daniela Elena Popescu and Octavian Proştean<br />

making step, migrati<strong>on</strong> step and management step. Thus, Figure 1 encompasses the proposed<br />

overall process and then each step is discussed separately.<br />

Figure 1: The management process of enterprise’s migrati<strong>on</strong> to IaaS<br />

2.1 Data analysis step<br />

Data analysis c<strong>on</strong>stitutes the initial step of the overall process taken by organizati<strong>on</strong>s to manage the<br />

migrati<strong>on</strong> to IaaS and it comprises: the analysis of cloud migrati<strong>on</strong> opportunities, the study of cloud<br />

adopti<strong>on</strong> barriers and the examinati<strong>on</strong> of current infrastructure used by the organizati<strong>on</strong>. In this<br />

process, enterprises have to c<strong>on</strong>sider both the assessment of risks and rewards even if an analysis of<br />

costs produced by implementing cloud services should be realized.<br />

Cloud rewards analysis<br />

According to Nati<strong>on</strong>al Institute of Standards and Technology (NIST), the Cloud c<strong>on</strong>cept is defined by<br />

five main characteristics: <strong>on</strong>-demand self-service, broad network access, resource pooling, rapid<br />

elasticity and measured service (Furlani, 2010), which c<strong>on</strong>stitutes the technical benefits:<br />

On-demand self-service: The customers could obtain the desired services from Cloud provider’s<br />

without any interacti<strong>on</strong> with the employees of the Cloud provider, because those services are<br />

requested <strong>on</strong>line by them.<br />

Broad network access: The devices that c<strong>on</strong>sumers use for access the Cloud vary (e.g. mobile<br />

ph<strong>on</strong>es, laptops and PDAs), which adds the flexibility capacity for SMEs (KPMG, 2011).<br />

Resource pooling: The customers in Cloud platform are multi-tenant. Even if the locati<strong>on</strong> of the<br />

resources is not actually known, the idea is that each of them requires over the Internet the<br />

exactly wanted resources (i.e. storage, processing, memory, network, virtual machines), by<br />

specifying locati<strong>on</strong> at a higher level of abstracti<strong>on</strong> (e.g. country, state or datacenter).<br />

Rapid elasticity: Any quantity of the capabilities could be purchased and released any time, which<br />

gives the elasticity feature in Cloud and it makes the clients to feel flexible in their opti<strong>on</strong>s. This<br />

capability is very c<strong>on</strong>venient for the SMEs that have different workload depending <strong>on</strong> several<br />

criteria: the m<strong>on</strong>ths of the year (Marst<strong>on</strong>, et al., 2011), the time of the day (Van Hoecke, et al.,<br />

2011). In this sense, Marst<strong>on</strong>, et al. (2011) emphasized the case of an internet photo website<br />

Smugmug that in December and January recorded computing workloads for five times higher<br />

than the usual <strong>on</strong>e. And Van Hoecke, et al. (2011) presented the internet company Nieuws.be,<br />

that has the field of activity in distributing nati<strong>on</strong>al and internati<strong>on</strong>al news <strong>on</strong> the web and reported<br />

an increased workload during the day hours by comparing with the decreased workload during<br />

the night hours.<br />

Measured Service is realized in cloud computing by m<strong>on</strong>itoring, c<strong>on</strong>trolling and reporting the<br />

usability of the resources. The customers are informed what they have to pay for c<strong>on</strong>suming<br />

those resources.<br />

Another cloud opportunity that enterprises should identify is the financial benefit (Khajeh-Hosseini, et<br />

al., 2011; KPMG, 2011). The reducti<strong>on</strong> of capability’s expenses is directed by the resource pooling<br />

cloud characteristic, in collaborati<strong>on</strong> with the elasticity capability of cloud providers, which optimize the<br />

cost usages. In additi<strong>on</strong>, the cost efficiency of cloud utilizati<strong>on</strong> is proved also by a case study realized<br />

170


Alina Mădălina L<strong>on</strong>ea, Daniela Elena Popescu and Octavian Proştean<br />

by Khajeh-Hosseini, Greenwood and Sommerville (2010), which calculates the system infrastructure<br />

costs involved over five year period for a company that maintain and provides IT soluti<strong>on</strong>s for the Oil<br />

& Gas industry. In this case study it was dem<strong>on</strong>strated that the costs of utilizing Amaz<strong>on</strong> EC2 cloud<br />

service are smaller than the costs of utilizing the traditi<strong>on</strong>al IT system and the company has the<br />

advantage of rapid elasticity feature of cloud and the enterprise’s in-house hosting costs are<br />

minimizing as well (e.g. electricity, cooling, off-site tape archiving). Thus, cloud computing records<br />

also a huge energy savings together with the cost benefit (Marst<strong>on</strong>, et al., 2011; Van Hoecke, et al.,<br />

2011). The IT investment for SMEs using cloud services is also reduced by the manageability of IT<br />

infrastructure of cloud providers which replace the work of IT support team provided by the in-house<br />

SMEs (KPMG, 2011). SME based cloud achieves also financial growth by changing the resp<strong>on</strong>sibility<br />

for the upgrades and compliance of the applicati<strong>on</strong>s, which will be the task of the cloud providers<br />

(KPMG, 2011).<br />

Furthermore, another cloud opportunity should be reflected <strong>on</strong>. This is the organizati<strong>on</strong>al growth of<br />

the enterprises, which will be realized by facilitating the sales and marketing department to create<br />

new products/services (Khajeh-Hosseini, Greenwood and Sommerville, 2010).<br />

Cloud risks analysis<br />

Companies which want to migrate to cloud services have also to identity the cloud adopti<strong>on</strong> risks and<br />

to c<strong>on</strong>sider how to manage the cloud adopti<strong>on</strong> barriers. The major c<strong>on</strong>cerns of enterprises are the<br />

security risks implied by the act of embedding their resources within the cloud computing envir<strong>on</strong>ment<br />

(Rittinghouse and Ransome, 2010). Hence, migrati<strong>on</strong> and integrati<strong>on</strong> phases of existing enterprise<br />

applicati<strong>on</strong> within the IaaS cloud services, should be deployed following a business migrati<strong>on</strong> plan,<br />

respectively a business disrupti<strong>on</strong> plan for the case that the migrati<strong>on</strong> process is causing the<br />

disrupti<strong>on</strong> of the business flow. All business managers, IT managers and IT vendors should cooperate<br />

in order to implement the business migrati<strong>on</strong>, respectively the business disrupti<strong>on</strong> plans (Saugatuck<br />

Technology, 2010). Disaster recovery solves handling the detecti<strong>on</strong> and preventi<strong>on</strong> of possible<br />

incidents and provides a Business C<strong>on</strong>tinuity Planning (BCP) which enhances the future growth of<br />

enterprises (CSA, 2009; CSCC, 2011). Although business migrati<strong>on</strong> plan, respectively business<br />

disrupti<strong>on</strong> plan cover a major area of security measures, encrypti<strong>on</strong> and key management usage in<br />

cloud computing is recognized as a core mechanism to protect resources (i.e. data in transit over<br />

networks, data at rest and data <strong>on</strong> backup media) (CSA, 2009). Even though the data is encrypted<br />

and a BCP exists, the measures inspected for securing the enterprise’s data are not sufficient for<br />

securing the cloud services and Identity and Access <strong>Management</strong> (IAM) branch has to be c<strong>on</strong>sidered,<br />

which secures the user identity of cloud computing services (RSA, 2009; CSA, 2009).<br />

Beside security, data governance for moving to cloud should comply with the specific enterprise’s<br />

regulatory requirements (e.g. physical locati<strong>on</strong> of data, data breach, pers<strong>on</strong>al data privacy, data<br />

destructi<strong>on</strong>, intellectual property, informati<strong>on</strong> ownership, law enforcement access, service availability)<br />

(CSCC, 2011). Hence, the health and financial sectors are distinguished with many regulatory<br />

restricti<strong>on</strong> of moving their data to cloud (Khajeh-Hosseini, Sommerville and Sriram, 2010). CSA<br />

(2009) recommends the ISO/IEC 27001/ 27002 certificati<strong>on</strong>s for certifying the informati<strong>on</strong> security<br />

management systems of providers, respectively the SAS 70 Type II for providing a reference for<br />

auditors. However, the applicability of data governance for SMEs sector should be increased, at the<br />

moment being poorly served (Caira and Begg, 2012).<br />

Additi<strong>on</strong>ally, at the stage when the company think to adopt cloud services, its employees are not<br />

prepared to deal with the cloud services. Thus, organizati<strong>on</strong>al issues are another challenges that<br />

should be perceived by enterprises (Heinle and Strebel, 2010), which will have to settle <strong>on</strong> the type of<br />

training activity: internal (i.e. by training their pers<strong>on</strong>al to use the cloud services) or external (i.e. by<br />

receiving temporarily or permanent external services) (CSCC, 2011). In this c<strong>on</strong>text, specific training<br />

should be realized in this area and in this way the employees will be aware about the changes<br />

produced by cloud transiti<strong>on</strong> and it will reduce their fake understanding of losing their jobs (Saugatuck<br />

Technology, 2010). Just the IT departments c<strong>on</strong>cerned <strong>on</strong>ly with the hardware and network support<br />

will suffer scaling down the number of jobs inside of those IT departments (Khajeh-Hosseini,<br />

Greenwood and Sommerville, 2010). Thus, this is another soluti<strong>on</strong> to manage the challenges that<br />

comes with the changes produced by cloud <strong>on</strong> the IT business department (Saugatuck Technology,<br />

2010). C<strong>on</strong>sequently with this organizati<strong>on</strong>al change produced by the cloud migrati<strong>on</strong>, the job<br />

satisfacti<strong>on</strong> of support engineers, sales & marketing staff and customer care staff is shrinking,<br />

171


Alina Mădălina L<strong>on</strong>ea, Daniela Elena Popescu and Octavian Proştean<br />

because the technical role of support engineers is switching to reporting issues and the satisfacti<strong>on</strong> of<br />

sales and marketing roles, respectively the satisfacti<strong>on</strong> of customer care depend up<strong>on</strong> the cloud<br />

based services (Khajeh-Hosseini, Greenwood and Sommerville, 2010).<br />

Assets examinati<strong>on</strong><br />

Further, the examinati<strong>on</strong> of current infrastructure used by the organizati<strong>on</strong> is useful because<br />

enterprises should know what type of bit architecture (i.e. 32 or 64 bit) have their hardware<br />

infrastructure and their operating systems (OSs), where are deployed their applicati<strong>on</strong>. Additi<strong>on</strong>ally,<br />

the business applicati<strong>on</strong>s should be investigated. This step of identifying these assets is required in<br />

order to prepare the migrati<strong>on</strong> process to an IaaS service compatible with the current infrastructure of<br />

the enterprise (Cisco Systems, Inc., 2010; Universität Osnabrück, 2012).<br />

2.2 Decisi<strong>on</strong> making step<br />

Decisi<strong>on</strong> making step implies the following decisi<strong>on</strong>s: what informati<strong>on</strong> should be moved into cloud<br />

and who will access the informati<strong>on</strong>, what Cloud Service Provider (CSP) the organizati<strong>on</strong> will choose<br />

and how the organizati<strong>on</strong> will manage the cloud services. We assumed that the cloud service type<br />

was chosen (i.e. IaaS) and the cloud deployment model was selected as well (i.e. public cloud).<br />

Choosing informati<strong>on</strong><br />

Enterprises should decide what informati<strong>on</strong> should be moved into cloud. This decisi<strong>on</strong> should be<br />

realized based <strong>on</strong> the cooperati<strong>on</strong> between the following departments: IT department and compliance<br />

department. The enterprise’s preoccupati<strong>on</strong> will establish a selecti<strong>on</strong> criteria of data and applicati<strong>on</strong><br />

preferred to migrate to cloud services, in order to assure c<strong>on</strong>fidentiality, integrity and availability<br />

requirements for the assets, based <strong>on</strong> the infrastructure examinati<strong>on</strong> and the cloud risks analysis<br />

(CSA, 2009; CSCC, 2011).<br />

Define service requirements<br />

In this sense being aware of the current infrastructure and applicati<strong>on</strong>s used <strong>on</strong> the company and<br />

being aware about what informati<strong>on</strong> should be moved into cloud, the enterprise can define service<br />

requirements for IaaS (Universität Osnabrück, 2012).<br />

Choosing CSP<br />

Choosing the right Cloud Service Provider (CSP) for enterprise will depend by the following criteria:<br />

cost efficiency, product strengths and market credibility. These three decisive factors were listed by<br />

Craig (2012) in the Enterprise <strong>Management</strong> Applicati<strong>on</strong> Report in order to emphasize the criteria of<br />

evaluating the Applicati<strong>on</strong> Performance <strong>Management</strong> of known soluti<strong>on</strong>s from the cloud marketplace,<br />

with the difference that instead of the market credibility it was used vendor strengths capability which<br />

includes a larger area of features. This paper suggests applying these determinants also for choosing<br />

the CSP, which will be discussed and evaluated for CSP (Figure 2).<br />

Figure 2: Choosing CSP<br />

Cost efficiency is <strong>on</strong>e of the decisive factors for choosing the CSP. This factor is composed by the<br />

following two sub-factors: cost advantage and deployment & administrati<strong>on</strong>. In terms of cost<br />

advantage, the modelling tool (from www.shopforcloud.com) described by Khajeh-Hosseini, et al.<br />

(2011) could help the costumers to deploy a cloud model by choosing the deployment elements<br />

172


Alina Mădălina L<strong>on</strong>ea, Daniela Elena Popescu and Octavian Proştean<br />

(i.e. server, storage and databases) from a variety list of cloud providers (i.e. Amaz<strong>on</strong> Web<br />

Services, Microsoft Azure, Rackspace). After defining the system requirements (i.e. the sec<strong>on</strong>d<br />

stage of the decisi<strong>on</strong> making step), enterprises may use this modelling tool in order to deploy the<br />

specified requirements. Hence, the tool produces a cost report based <strong>on</strong> selecti<strong>on</strong> of its<br />

computati<strong>on</strong>al resource usage patterns. This modelling tool is a free web interface, helpful for<br />

comparing the pricing schemes around cloud providers. Nevertheless, the cost reports calculates<br />

<strong>on</strong>ly the costs involved in deployment of a system based infrastructure, where it can be added<br />

additi<strong>on</strong>al costs (e.g. 3 rd party plugin to m<strong>on</strong>itor costs – Cloudability.com, 3 rd party platform to<br />

manage cloud resources – RightScale cloud <strong>Management</strong>). Additi<strong>on</strong>al costs may include license<br />

costs, training/c<strong>on</strong>sulting services costs, expenditure of time c<strong>on</strong>suming for employees who<br />

migrate to cloud services etc (Universität Osnabrück, 2012; Khajeh-Hosseini, et al., 2011).<br />

However, besides cost advantage another decisive factor which proves the cost efficiency is the<br />

deployment and administrati<strong>on</strong> analysis (i.e. ease of deployment, support and services, ease of<br />

administrati<strong>on</strong>).<br />

Product strength analysis provides informati<strong>on</strong> about the architecture and integrati<strong>on</strong> features,<br />

respectively about the functi<strong>on</strong>ality of CSP.<br />

Market credibility strengthens the enterprise’s decisi<strong>on</strong> about choosing the CSP, by analyzing the<br />

reputati<strong>on</strong> of CSP <strong>on</strong> the market.<br />

Choosing management tools<br />

Cloud management is a subject approached by researchers in the community and this can be<br />

observed by the big number of third party cloud management providers (i.e. RightScale, enStratus,<br />

IMOD Kaavo, CloudWatch, Scalr, Tapin, Cloudkick). These third party cloud management tools are<br />

commercial versi<strong>on</strong>s, used in special by organizati<strong>on</strong>s which want to manage their cloud<br />

infrastructure. Thus, enterprises should select <strong>on</strong>e of these commercial versi<strong>on</strong>s. In the Enterprise<br />

<strong>Management</strong> Applicati<strong>on</strong> Report, Craig (2012) emphasizes the criteria of evaluating the Applicati<strong>on</strong><br />

Performance <strong>Management</strong> of known soluti<strong>on</strong>s from the cloud marketplace. According with Craig<br />

(2012) three criteria of selecti<strong>on</strong>s is c<strong>on</strong>sidered in their survey: cost efficiency, product strengths and<br />

vendor strengths. The results of the survey which define the vendor strengths include several<br />

categories, namely: visi<strong>on</strong>, strategy, financial strength, research development and market credibility of<br />

vendors. For choosing the management tools, this paper proposes the first two criteria of selecti<strong>on</strong>:<br />

cost efficiency and product strengths from the report provided by Craig (2012) and instead of all<br />

vendor strengths elements, <strong>on</strong>ly the market credibility feature of the vendor will be evaluated.<br />

The appraisal of cost efficiency should include the following objectives: cost advantage and<br />

deployment & administrati<strong>on</strong> analysis. Whilst the cost advantage is determined by price, licensing<br />

and maintenance costs of the management tool, the deployment and administrati<strong>on</strong> analysis is<br />

made to dem<strong>on</strong>strate the ease of deployment (i.e. time to deploy, packaging requirements, staff<br />

training, disrupti<strong>on</strong> minimizati<strong>on</strong>), a high vendor’s customer support and the ease of<br />

administrati<strong>on</strong> (i.e. <strong>on</strong>going administrati<strong>on</strong>, update process, testing/migrati<strong>on</strong>).<br />

The investigati<strong>on</strong> of product strengths is another selecti<strong>on</strong>’s criteri<strong>on</strong> of management tools and it<br />

should reveal an analysis of architecture and integrati<strong>on</strong> categories, respectively the analysis of<br />

their functi<strong>on</strong>ality.<br />

The vendor’s market credibility feature will review the reputati<strong>on</strong> of the vendors <strong>on</strong> the cloud<br />

marketplace, with the purpose of enhancing the decisi<strong>on</strong>, after evaluating the cost efficiency and<br />

the product strengths.<br />

2.3 Migrati<strong>on</strong> step<br />

Migrati<strong>on</strong> step is the effective moving stage of enterprise’s assets into cloud services. This step<br />

includes 2 activities: developing the Service Level Agreement (SLA) and implementing cloud.<br />

Developing SLA<br />

The Service Level Agreement (SLA) is a document that should be compulsory d<strong>on</strong>e between the<br />

cloud provider and the customer, in order to obtain and to maintain a clear aspect over the rights and<br />

the resp<strong>on</strong>sibilities of each party. This document is relevant for avoiding c<strong>on</strong>flict that could occur<br />

173


Alina Mădălina L<strong>on</strong>ea, Daniela Elena Popescu and Octavian Proştean<br />

during the c<strong>on</strong>tract, because it should specify a wide range of issues and the remedies and warranties<br />

of them (Kandukuri, Paturi and Rakshit, 2009).<br />

The c<strong>on</strong>tent of a typical service level agreement (Kandukuri, Paturi and Rakshit, 2009) is representing<br />

in Figure 3.<br />

Figure 3: Typical service level agreement c<strong>on</strong>tent<br />

Definiti<strong>on</strong> of Services is part of the SLA document, where the services are defined and described<br />

using detailed informati<strong>on</strong>, for create a good understanding over exactly what is being delivered. After<br />

the services are defined in SLA, then another secti<strong>on</strong> of it (Performance <strong>Management</strong>) should c<strong>on</strong>tain<br />

aspects of m<strong>on</strong>itoring and measuring the service performance. Including benchmarks, targets and<br />

metrics in the requirements of SLA, the both parties of the agreement will be involved in m<strong>on</strong>itoring<br />

the performance of the services. A reliable management arises with the agreement, because relating<br />

and discussing the management problems will also be a part of the c<strong>on</strong>tract. Problem <strong>Management</strong><br />

embraces the methods for preventing and combating the incidents (Kandukuri, Paturi and Rakshit,<br />

2009).<br />

The customer duties and resp<strong>on</strong>sibilities relate the obligati<strong>on</strong>s of the cloud’s customers, because in<br />

the agreement document each party plays its role and have its own resp<strong>on</strong>sibility. While the<br />

customers have resp<strong>on</strong>sibilities also the provider of cloud should have warranties and remedies<br />

(Kandukuri, Paturi and Rakshit, 2009).<br />

The SLA also should provide security features, creating c<strong>on</strong>trol access to the informati<strong>on</strong> established<br />

by the customers and including the client’s security policies and procedures that must be performed<br />

by suppliers. Beside the security features, both parties should include in the agreement document a<br />

disaster recovery and business c<strong>on</strong>tinuity feature, because if an unplanned disaster happen, the<br />

customer should have the guaranty of safeguarding the data and the cloud provider should thing to<br />

keep its clients, by assuring the disaster recovery plan (Kandukuri, Paturi and Rakshit, 2009; CSCC,<br />

2011).<br />

The final chapter in a typical service level agreement is the terminati<strong>on</strong> secti<strong>on</strong>, which should have the<br />

following topics: terminati<strong>on</strong> at end of initial term; terminati<strong>on</strong> for c<strong>on</strong>venience; terminati<strong>on</strong> for cause<br />

and payments <strong>on</strong> terminati<strong>on</strong> (Kandukuri, Paturi and Rakshit, 2009).<br />

Implementing Cloud c<strong>on</strong>stitutes the effective migrati<strong>on</strong> of the enterprise’s informati<strong>on</strong> to the cloud<br />

service. This step will deploy the system using the CSP capabilities and the system requirements<br />

previously defined (i.e. phase 2 of decisi<strong>on</strong> making step), by migrating the informati<strong>on</strong> (i.e. phase 1 of<br />

decisi<strong>on</strong> making step) to the cloud service.<br />

2.4 <strong>Management</strong> step<br />

After migrating to cloud services, enterprises will manage the deployed cloud, using two management<br />

functi<strong>on</strong>s: business and operati<strong>on</strong>al. Business management functi<strong>on</strong>, also called administrative group<br />

by DMTF (2010b) guarantees the following business supports: customer management, c<strong>on</strong>tract<br />

management, inventory management, accounting and billing, pricing and ratings, metering and SLA<br />

management (DMTF, 2010a; Hogan, et al., 2011).<br />

174


Alina Mădălina L<strong>on</strong>ea, Daniela Elena Popescu and Octavian Proştean<br />

The sec<strong>on</strong>d management functi<strong>on</strong> called operati<strong>on</strong>al management functi<strong>on</strong> or resource management<br />

group by DMTF (2010b), is handling the provisi<strong>on</strong>ing/c<strong>on</strong>figurati<strong>on</strong> operati<strong>on</strong>s and<br />

portability/interoperability operati<strong>on</strong>s (Hogan, et al., 2011; DMTF, 2010a).<br />

3. Related work<br />

The steps of the proposed process have been separately approached by researchers in the area. It<br />

can be observed that the most desired topics of scientists related with this paper are the study of<br />

cloud benefits and the study of cloud barriers. Thus, the detailed reports provided by Cloud Security<br />

Alliance (CSA, 2009) and the <str<strong>on</strong>g>European</str<strong>on</strong>g> Network and Informati<strong>on</strong> Security Agency (Catteddu and<br />

Hogben, 2009) c<strong>on</strong>tain assessments of security risks and benefits of using cloud services. Kourik<br />

(2011) present an overview of the risk assessment instruments for SMEs developed by Open Group,<br />

CSA, Nati<strong>on</strong>al Institute of Standards and Technology (NIST), ENISA. Additi<strong>on</strong>ally, white papers<br />

produced by CSPs offer decisi<strong>on</strong> maker’s recommendati<strong>on</strong>s for enterprises (Chappell, 2009; Varia,<br />

2010), which represent the marketing tools.<br />

Moreover, the practical guide provided by Cloud Standards Customer Council (CSCC, 2011) offers a<br />

series of steps recommended to be adopted by customers, with the purpose to ensure a successful<br />

cloud deployment and to make visible the differences produced by the size and the IT maturity level of<br />

the enterprises. This idea c<strong>on</strong>ducted at the c<strong>on</strong>clusi<strong>on</strong> that start-up SMEs prefer to adopt the public<br />

cloud instead of private cloud, while SMEs with existing infrastructure prefer to adopt hybrid cloud<br />

(Van Hoecke, et al., 2011). However, the steps described in (CSCC, 2011) are approached differently<br />

comparing with the steps of the overall process discussed in this paper.<br />

Khajeh-Hosseini, et al. (2011) created a benefits and risks assessment tool of using public cloud<br />

IaaS, by identifying the following categories: organizati<strong>on</strong>al, legal, security, technical and financial<br />

risks and the technical, organizati<strong>on</strong>al and financial benefits. Khajeh-Hosseini, Sommerville and<br />

Sriram (2010) looked into the research challenges for cloud computing for an enterprise perspective:<br />

organizati<strong>on</strong>al changes, ec<strong>on</strong>omic and organizati<strong>on</strong>al implicati<strong>on</strong>s and the security, legal and privacy<br />

issues. In additi<strong>on</strong>, the cost efficiency of cloud utilizati<strong>on</strong> is proved by a case study realized by Khajeh-<br />

Hosseini, Greenwood and Sommerville (2010), which calculates the system infrastructure costs<br />

involved over five year period for a company that maintain and provides IT soluti<strong>on</strong>s for the Oil & Gas<br />

industry.<br />

The study of cloud risks in our paper emphasizes three issues: security issues debated in<br />

(Rittinghouse and Ransome, 2010), data governance discussed in (Caira and Begg, 2012; CSCC, 2011;<br />

CSA, 2009; Khajeh-Hosseini, Sommerville and Sriram, 2010) and organizati<strong>on</strong>al issues reviewed in<br />

(Heinle and Strebel, 2010), with their corresp<strong>on</strong>ding measures deliberated in the following papers:<br />

(Saugatuck Technology, 2010; CSA, 2009; CSCC, 2011; RSA, 2009; Khajeh-Hosseini, Sommerville<br />

and Sriram, 2010; Heinle and Strebel, 2010; Khajeh-Hosseini, Greenwood and Sommerville, 2010;<br />

Martens and Teuteberg, 2011).<br />

Cisco Systems, Inc. (2010) presented the migrati<strong>on</strong> plan of enterprise applicati<strong>on</strong>s to the cloud, by<br />

analyzing which cloud services is suited for the selected applicati<strong>on</strong>. Because in our paper we<br />

assumed that the cloud service chosen is IaaS, the interest from (Cisco Systems, Inc., 2010) is IaaS<br />

service. According to Cisco Systems, Inc. (2010) for deciding about IaaS service the enterprise<br />

should examine its infrastructure, idea supported also by Universität Osnabrück (2012).<br />

Decisi<strong>on</strong> making step is based <strong>on</strong> the analysis step. The asset decisi<strong>on</strong> was discussed in (CSA, 2009;<br />

CSCC, 2011). Our paper suggests applying three determinants for choosing the CSP: cost efficiency,<br />

product strengths and market credibility, decisive factors that were listed by Craig (2012) in the<br />

Enterprise <strong>Management</strong> Applicati<strong>on</strong> Report in order to emphasize the criteria of evaluating the<br />

Applicati<strong>on</strong> Performance <strong>Management</strong> of known soluti<strong>on</strong>s from the cloud marketplace, with the<br />

difference that instead of the market credibility it was used vendor strengths capability which includes<br />

a larger area of features. The same decisive factors were recommended for choosing the<br />

management tools of cloud services.<br />

Kandukuri, Paturi and Rakshit (2009) and CSCC (2011) c<strong>on</strong>stitute the sources for the stage of<br />

developing the Service Level Agreement (SLA). With respect to the management step, two<br />

management functi<strong>on</strong>s were discussed: business and operati<strong>on</strong>al (DMTF, 2010a; DMTF, 2010b;<br />

Hogan, et al., 2011).<br />

175


4. C<strong>on</strong>clusi<strong>on</strong>s<br />

Alina Mădălina L<strong>on</strong>ea, Daniela Elena Popescu and Octavian Proştean<br />

Enterprises adhere to cloud computing technology, which is subject to a c<strong>on</strong>tinuous development.<br />

Simultaneously with the increased number of enterprises that adopt cloud computing, the challenges<br />

of enterprises to exploit cloud for their business objectives are growing as well. Thus, companies go<br />

through a holistic process in order to manage the implemented cloud services. This paper discussed<br />

the overall process taken by Small and medium-sized enterprises (SMEs) to manage the migrati<strong>on</strong> of<br />

their applicati<strong>on</strong>s to Infrastructure-as-a-Service (IaaS), which includes a collecti<strong>on</strong> of the following<br />

interrelated activities: data analysis step, decisi<strong>on</strong> making step, migrati<strong>on</strong> step and management step.<br />

The presented process is a complex approach which reveals the migrati<strong>on</strong> of SMEs to the cloud<br />

services, and this process was created based <strong>on</strong> several criteria that SMEs should analyze in order to<br />

make a suitable decisi<strong>on</strong>. Thus, in order to realize the replacement procedure of the existing services<br />

from the traditi<strong>on</strong>al SMEs with the cloud services, the following patterns were used in the c<strong>on</strong>cepti<strong>on</strong> of<br />

the outsourcing procedure: the SME’s business model, the size of the organizati<strong>on</strong>, the IT maturity<br />

level of SME, the SME’s sector, the SME’s informati<strong>on</strong>, the establishment of the periods when the SME<br />

records intense and regular workload, the skills and knowledge of SME about cloud services, the<br />

Cloud Service Provider (CSP) market and the cloud management tools.<br />

The proposed process improves the efficiency, quality and capacity management of enterprises to<br />

move their data and applicati<strong>on</strong>s into cloud. Furthermore, it decreases the enterprise’s expenditure for<br />

decisi<strong>on</strong> making as regards transiti<strong>on</strong> into cloud. Due to the innovati<strong>on</strong> brought by cloud computing<br />

technology which improves the Informati<strong>on</strong> and Communicati<strong>on</strong> Technology (ICT), the cloud adopti<strong>on</strong><br />

by SMEs maintains the competitiveness. The limitati<strong>on</strong> of this paper is that it is a qualitative research,<br />

which does not provide a case study for evaluating the described process. However, this will be part of<br />

our future work.<br />

Acknowledgements<br />

This work was partially supported by the strategic grant POSDRU/88/1.5/S/50783, Project ID50783<br />

(2009), co-financed by the <str<strong>on</strong>g>European</str<strong>on</strong>g> Social Fund – Investing in People, within the Sectoral<br />

Operati<strong>on</strong>al Programme Human Resources Development 2007-2013.<br />

References<br />

Adam, F., and O'Doherty, P., (2000). "Less<strong>on</strong>s from enterprise resource planning implementati<strong>on</strong>s in Ireland -<br />

towards smaller and shorter ERP projects," Journal of Informati<strong>on</strong> Technology (15:4), pp 305-316.<br />

Begg, C. and Caira, T., (2012). “Exploring the SME Quandary: Data Governance in Practise in the Small to<br />

Medium-Sized Enterprise Sector”. The Electr<strong>on</strong>ic Journal Informati<strong>on</strong> Systems Evaluati<strong>on</strong>, Volume 15,<br />

Issue 1, pp. 01-12.<br />

Catteddu, D. and Hogben, G. (2009). “Cloud Computing: benefits, risks and recommendati<strong>on</strong>s for informati<strong>on</strong><br />

security”.<br />

Cisco Systems, Inc. (2010).”Planning the Migrati<strong>on</strong> of Enterprise Applicati<strong>on</strong>s to the Cloud”. [<strong>on</strong>line], Cisco White<br />

Paper,<br />

http://www.cisco.com/en/US/services/ps2961/ps10364/ps10370/ps11104/Migrati<strong>on</strong>_of_Enterprise_Apps_to<br />

_Cloud_White_Paper.pdf.<br />

Chappell, D. (2009). “Winodws Azure and ISVs: A Guide for Decisi<strong>on</strong> Makers”.<br />

Craig, J. (2012). “EMA Radar TM for Applicati<strong>on</strong> Performance <strong>Management</strong> (APM) for Cloud Services: Q1 2012”.<br />

Enterprise <strong>Management</strong> Associates (EMA).<br />

CSA, (2009). “Security Guidance for Critical Areas of Focus in Cloud Computing V2.1”, [<strong>on</strong>line], Cloud Security<br />

Alliance, https://cloudsecurityalliance.org/csaguide.pdf.<br />

CSCC, (2011). “Practical Guide to Cloud Computing Versi<strong>on</strong> 1.2”, [<strong>on</strong>line], Cloud Standards Customer Council,<br />

http://www.isaca.org/Groups/Professi<strong>on</strong>alEnglish/cloudcomputing/GroupDocuments/CSCC_PG2CCv1_2.pd<br />

f.<br />

Dai, W. (2009). “The Impact of Emerging Technologies <strong>on</strong> Small and Medium Enterprises”. Journal of Business<br />

Systems, Governance and Ethics, Vol. 4, No. 4, pp. 53-60.<br />

Devos, J., Van Landeghem, H. and Deschoolmeester, D. (2012). SMEs and IT: Evidence for a Market for<br />

“Lem<strong>on</strong>s”. The Electr<strong>on</strong>ic Journal Informati<strong>on</strong> Systems Evaluati<strong>on</strong>, Volume 15, Issue 1, pp. 25-35.<br />

DMTF, (2010a). “Use Cases and Interacti<strong>on</strong> for managing clouds.” [<strong>on</strong>line], White paper from the Open Cloud<br />

Standards Incubator, Versi<strong>on</strong> 1.0.0. Distributed <strong>Management</strong> Task Force Inc.,<br />

http://www.dmtf.org/sites/default/files/standards/documents/DSP-IS0103_1.0.0.pdf.<br />

DMTF, (2010b). “DMTF Architecture for managing clouds”, [<strong>on</strong>line], Distributed <strong>Management</strong> Task Force, Inc.,<br />

http://www.dmtf.org/sites/default/files/standards/documents/DSP-IS0102_1.0.0.pdf.<br />

176


Alina Mădălina L<strong>on</strong>ea, Daniela Elena Popescu and Octavian Proştean<br />

Heinle, C. and Strebel, J.. (2010). “IaaS adopti<strong>on</strong> determinants in enterprises”. In Proceedings of the 7th<br />

internati<strong>on</strong>al c<strong>on</strong>ference <strong>on</strong> Ec<strong>on</strong>omics of grids, clouds, systems, and services (GECON'10), J. Altmann and<br />

Omer F. Rana (Eds.). Springer-Verlag, Berlin, Heidelberg, 93-104.<br />

Hogan, M., Liu, F., Sokol, A. and T<strong>on</strong>g, J. (2011). “NIST Cloud Computing Standards Roadmap – Versi<strong>on</strong> 1.0”.<br />

NIST Nati<strong>on</strong>ale Institute of Standards and Technologies.<br />

Kandukuri, B. R., Paturi, R.V. and Rakshit, A. (2009). “Cloud Security Issues”. In: IEEE Internati<strong>on</strong>al <str<strong>on</strong>g>C<strong>on</strong>ference</str<strong>on</strong>g><br />

<strong>on</strong> Services Computing. Bangalore, 21-25 September 2009, pp. 517-520.<br />

Khajeh-Hosseini A., Sommerville I., Bogaerts J., Teregowda P.. (2011). “Decisi<strong>on</strong> Support Tools for Cloud<br />

Migrati<strong>on</strong> in the Enterprise”. IEEE 4 th Internati<strong>on</strong>al <str<strong>on</strong>g>C<strong>on</strong>ference</str<strong>on</strong>g> <strong>on</strong> Cloud Computing (CLOUD 2011),<br />

Washingt<strong>on</strong> DC, USA.<br />

Khajeh-Hosseini A., Greenwood D. and Sommerville I. (2010). “Cloud Migrati<strong>on</strong>: A Case Study of Migrating an<br />

Enterprise IT System to IaaS”. IEEE 3 rd Internati<strong>on</strong>al <str<strong>on</strong>g>C<strong>on</strong>ference</str<strong>on</strong>g> <strong>on</strong> Cloud Computing (CLOUD 2010).<br />

Miami, USA.<br />

Khajeh-Hosseini A., Sommerville I. and Sriram I. (2010). “Research Challenges for Enterprises Cloud<br />

Computing”. LSCITS Technical Report.<br />

Kourik, J.L., (2011). “For Small and Medium Size Enterprises (SME) Deliberating Cloud Computing: A Proposed<br />

Approach”. Proceedings of the <str<strong>on</strong>g>European</str<strong>on</strong>g> Computing <str<strong>on</strong>g>C<strong>on</strong>ference</str<strong>on</strong>g>, ISBN: 978-960-474-297-4, pp. 216-221.<br />

KPMG, (2011). “The Cloud Changing the Business Ecosystem”. [<strong>on</strong>line]<br />

http://www.kpmg.com/IN/en/IssuesAndInsights/ThoughtLeadership/The_Cloud_Changing_the_Business_E<br />

cosystem.pdf.<br />

Furlani, C.M. (2010). “Cloud Computing: Benefits and Risks of Moving Federal IT into the Cloud”, [<strong>on</strong>line],<br />

Nati<strong>on</strong>al Institute of Standards and Technology (NIST),<br />

http://www.nist.gov/director/ocla/testim<strong>on</strong>y/upload/Cloud-Computing-testim<strong>on</strong>y-FINAL-with-Bio.pdf.<br />

Levy, M., and Powell, P. (2004).”Strategies for Growth in SMEs: The Role of Informati<strong>on</strong> and Informati<strong>on</strong><br />

Systems”. Elsevier, Oxford.<br />

Marst<strong>on</strong>, S., et al. (2011). “Cloud Computing – The business perspective”. Decisi<strong>on</strong> Support Systems, 51, pp.<br />

176-189.<br />

Martens, B. and Teuteberg, F. (2011). ”Risk and Compliance <strong>Management</strong> for Cloud Computing Services:<br />

Designing a Reference Model”. In Proceedings of the Seventeenth Americas <str<strong>on</strong>g>C<strong>on</strong>ference</str<strong>on</strong>g> of Informati<strong>on</strong><br />

Systems, Detroit, Michigan, August 4 th -7 th .<br />

Misra, S.C. and M<strong>on</strong>dal, A. (2011). “Identificati<strong>on</strong> of a company's suitability for the adopti<strong>on</strong> of cloud computing<br />

and modelling its corresp<strong>on</strong>ding Return <strong>on</strong> Investment”. Mathematical and Computer Modelling, Volume 53,<br />

Issues 3-4, pp. 504-521.<br />

Universität Osnabrück, (2012). “Total Cost of Ownership Calculator for Cloud Computing Services”, [<strong>on</strong>line],<br />

http://www.cloudservicemarket.info/tools/tco.aspx.<br />

Rittinghouse, J. W. and Ransome, J.F. (2010). Cloud Computing Implementati<strong>on</strong>, <strong>Management</strong> and Security.<br />

Boca Rat<strong>on</strong>: CRC Press.<br />

RSA, (2009). “The Role of Security in Trustworthy Cloud Computing”, [<strong>on</strong>line], White Paper,<br />

http://www.emc.com/collateral/about/investor-relati<strong>on</strong>s/9921_CLOUD_WP_0209_lowres.pdf.<br />

Saugatuck Technology Inc. (2010). “Stepping Up to the Cloud: Managing Changes and Migrati<strong>on</strong> for Mid-Sized<br />

Business”, [<strong>on</strong>line], http://fm.sap.com/data/UPLOAD/files/Saugatuck-Stepping_Up_to_the_Cloud-<br />

Managing_Changes_and_Migrati<strong>on</strong>_for_Mid-sized_Business.pdf.<br />

Sharma, M., et al. (2010). “Scope of Cloud Computing for SMEs in India”. Journal of Computing, Volume 2, Issue<br />

5, ISSN: 2151-9617.<br />

Van Hoecke, S., et al. (2011). “Efficient <strong>Management</strong> of Hybrid Clouds”. Cloud Computing 2011: The Sec<strong>on</strong>d<br />

Internati<strong>on</strong>al <str<strong>on</strong>g>C<strong>on</strong>ference</str<strong>on</strong>g> <strong>on</strong> Cloud Computing, Grids and Virtualizati<strong>on</strong>, pp. 167-172.<br />

Varia, J. (2010). “Migrating your Existing Applicati<strong>on</strong>s to the AWS Cloud”. Amaz<strong>on</strong> Web Services.<br />

Weinman, J. (2011). “Axiomatic Cloud Theory”. [<strong>on</strong>line],<br />

http://www.joeweinman.com/Resources/Joe_Weinman_Axiomatic_Cloud_Theory.pdf.<br />

177


Sustainable Enterprise Architecture: A Three-Dimensi<strong>on</strong>al<br />

Framework for <strong>Management</strong> of Architectural Change<br />

Thanos Magoulas, Aida Hadzic, Ted Saarikko and Kalevi Pessi<br />

Department of Applied IT, Gothenburg University, Göteborg, Sweden<br />

thanos.magoulas@gu.se<br />

hadzic@chalmers.se<br />

ted.saarikko@ituniv.se<br />

kalevi.pessi@gu.se<br />

Abstract: Despite advances in informati<strong>on</strong> technology, the modern enterprise finds itself struggling to satisfy its<br />

need for pertinent informati<strong>on</strong>. Faced with the challenge of matching internal resources to external demands,<br />

more and more organizati<strong>on</strong>s turn to Enterprise Architecture (EA) for guidance. Yet all too often change efforts in<br />

relati<strong>on</strong> to EA are approached with a simplistic- or technical perspective that limits future development. We must<br />

therefore seek a more appropriate means to facilitate purposeful change and ensure a sustainable EA that is able<br />

to accommodate the complexities that arise when dealing with complex change efforts. In resp<strong>on</strong>se to the need<br />

for better understanding of sustainability in Enterprise Architecture, we propose a tentative three-dimensi<strong>on</strong>al<br />

framework for change c<strong>on</strong>sisting of three dimensi<strong>on</strong>s: Perspectives <strong>on</strong> change, levels of change and types of<br />

change. The perspective <strong>on</strong> change is a reflecti<strong>on</strong> of the paradigm up<strong>on</strong> which the change effort is based. Two<br />

extreme views are hard systems thinking and soft systems thinking. The level of change describes its delineati<strong>on</strong>.<br />

Depending <strong>on</strong> scope, the change effort may be c<strong>on</strong>sidered local, structural or inter-organizati<strong>on</strong>al in nature. The<br />

nature of change signifies the extent to which an enterprise departs from existing practices. We may refer to<br />

changes as incremental, transformati<strong>on</strong>al or reorientati<strong>on</strong>-based depending <strong>on</strong> their magnitude. The relevance<br />

and usefulness of these three dimensi<strong>on</strong>s has been validated through seminars, workshops or advanced courses<br />

held with over 30 healthcare professi<strong>on</strong>als. While we feel that this framework is a step in the right directi<strong>on</strong>, there<br />

is still much work to be d<strong>on</strong>e in this area. We therefore call for a deeper discourse and further research into<br />

sustainability in Enterprise Architecture.<br />

Keywords: enterprise architecture, change, sustainability<br />

1. Introducti<strong>on</strong><br />

Faced with increasing instability and diversity in terms of operati<strong>on</strong>al c<strong>on</strong>text as well as internal<br />

structure, many organizati<strong>on</strong>s turn to Enterprise Architecture (EA) for guidance. To this end, much<br />

effort has been spent establishing suitable tax<strong>on</strong>omies (Sowa & Zachman, 1992), describing<br />

enterprise modelling (IFIP/IFAC, 2003), and developing methodologies for modifying architecture (The<br />

Open Group, 2009, p. 49-65). C<strong>on</strong>siderably less has been said in regards to its sustainability – the<br />

ability to either absorb c<strong>on</strong>textual changes or adapt in order to accommodate them. Given that the<br />

impetus for adopting EA is to manage change and complexity, it is remarkable that <strong>on</strong>e would<br />

overlook the need to maintain and develop the architecture itself. In a c<strong>on</strong>ference moderated by the<br />

Open Group, their vice president of skills and capabilities said in reference to architecture and<br />

transformati<strong>on</strong>: “My positi<strong>on</strong> is that they’re two separate entities” (Gardner, 2012). A clue into the root<br />

of this peculiar c<strong>on</strong>tradicti<strong>on</strong> may be the technical focus of the topical literature. Normative<br />

approaches to EA, often referred to as frameworks, usually delineate EA into layers such as business,<br />

applicati<strong>on</strong>, informati<strong>on</strong>, data, technology et cetera. Given the focus <strong>on</strong> IT artefacts, it is perhaps no<br />

w<strong>on</strong>der that EA is treated much in the same way as hardware or software; use until failure or<br />

obsolescence, then replace. This manner of change has been referred to as disc<strong>on</strong>tinuous (Nadler &<br />

Tushman, 1995), radical (Orlikowski, 1993) or episodic (Weick & Quinn, 1999). However, given the<br />

sheer amount of time, m<strong>on</strong>ey and expertise that goes into establishing an Enterprise Architecture,<br />

applying such an approach would be extremely wasteful. Hence, <strong>on</strong>ce the architecture is in place – it<br />

often stays in place, gradually diverging from the ephemeral envir<strong>on</strong>ment until it has become outdated<br />

and more of a burden than a bo<strong>on</strong> to the enterprise. Again, this approach has received its fair share of<br />

attenti<strong>on</strong> under the guise of incremental- (Nadler & Tushman, 1995; Orlikowski, 1993) or c<strong>on</strong>tinuous<br />

change (Weick & Quinn, 1999).<br />

It would seem that we need a more balanced view if we are to understand and discuss change efforts<br />

within the realm of Enterprise Architecture. A sustainable approach that allows the architecture to<br />

develop as needed, when needed by the stakeholders of the enterprise (Kay, 1993).<br />

In this paper, we introduce three dimensi<strong>on</strong>s of change which we have derived from extant literature<br />

in the fields of organizati<strong>on</strong>al- and informati<strong>on</strong> systems research. As such, the framework is based <strong>on</strong><br />

178


Thanos Magoulas et al.<br />

a deductive approach that has been validated through seminars, workshops or advanced courses<br />

held with more than 30 Swedish healthcare professi<strong>on</strong>als. Most of the participants have worked with<br />

IT in a healthcare c<strong>on</strong>text for more than 15 years. We will also utilize their experiences as a means to<br />

exemplify practical instances of different change efforts.<br />

2. C<strong>on</strong>cept of change in enterprise architecture<br />

There are two topics that are relevant to our discourse <strong>on</strong> sustainability in Enterprise Architecture:<br />

Sustainability and change.<br />

The aptness of utilizing sustainability in a perpetually changing business envir<strong>on</strong>ment is discussed by<br />

Aier (2004). Noting the general dearth of relevant (i.e. business-oriented) literature, he approaches<br />

the issue by means of mapping extant literature <strong>on</strong> sustainability <strong>on</strong>to the c<strong>on</strong>struct of EA.<br />

Ascertaining that c<strong>on</strong>stant change gives rise to frequent reorganizati<strong>on</strong>s, he proposes modularizati<strong>on</strong><br />

as a viable and fiscally rati<strong>on</strong>al means to sustain the architecture in the face of envir<strong>on</strong>mental<br />

turbulence.<br />

In a subsequent paper, Aier and Schoenherr (2005) further pursued the potential of developing<br />

sustainability through modularizati<strong>on</strong> by means of Enterprise Applicati<strong>on</strong> Integrati<strong>on</strong> (EAI). C<strong>on</strong>ducting<br />

an empirical study covering approximately 60 EAI users, c<strong>on</strong>sultants and vendors, they c<strong>on</strong>clude that<br />

while EAI shows significant promise, its utilizati<strong>on</strong> is presently restricted to artifacts. This, of course,<br />

limits its potential c<strong>on</strong>tributi<strong>on</strong> towards sustainability – or any other n<strong>on</strong>-technical aspect of EA.<br />

Kluge, Dietzsch and Rosemann (2006) present a case study of EA value based <strong>on</strong> the DeL<strong>on</strong>e &<br />

McLean model of IS success. In their view, a lack of n<strong>on</strong>-IT stakeholder acceptance is a major<br />

impediment to sustainable implementati<strong>on</strong> of EA. Although limited in scope, their findings suggest that<br />

improving stakeholder awareness serves to promote purposeful architecture change efforts. By<br />

ensuring a perceived value of EA, stakeholder involvement brings about a more holistic<br />

understanding and thus greater c<strong>on</strong>formance to the needs of the enterprise.<br />

Fischer, Aier and Winter (2007) explore the possibilities of utilizing EA modelling in order to effect<br />

purposeful change whilst maintaining IT-business alignment. The primary role of EA is perceived as<br />

<strong>on</strong>e of documentati<strong>on</strong> in order to increase awareness of relevant structures and processes. Hence,<br />

EA may serve enterprise transformati<strong>on</strong> by means of ensuring the availability of pertinent informati<strong>on</strong>.<br />

Fischer et al. c<strong>on</strong>clude that integrated and updated models not <strong>on</strong>ly provide valuable informati<strong>on</strong>, but<br />

also serve to dem<strong>on</strong>strate the value of EA to stakeholders.<br />

Ren and Lyytinen (2008) as well as Schelp and Aier (2009) propose Service Oriented Architecture<br />

(SOA) as an agent for achieving an agile EA. Ren and Lyytinen scarcely move bey<strong>on</strong>d the technical<br />

domain, c<strong>on</strong>tent with providing s number of definiti<strong>on</strong>s of SOA and pointing out that several am<strong>on</strong>g<br />

them acknowledge business requirements and user needs. They c<strong>on</strong>clude that while SOA has the<br />

potential to enable a higher degree of agility, practical implementati<strong>on</strong>s are often limited to certain<br />

technical envir<strong>on</strong>ments – a state of affairs which diametrically opposes the noti<strong>on</strong> of SOA as<br />

independent of platform.<br />

Schelp and Aier (2009) c<strong>on</strong>duct a case-study whereby they ascertain the potential of SOA to improve<br />

enterprise agility. Their findings suggest that SOA does indeed have a positive impact <strong>on</strong> agility in<br />

terms of shortening time-to-market of architecture comp<strong>on</strong>ents. Other potential benefits, such as<br />

reusability and flexibility, are highly dependent <strong>on</strong> c<strong>on</strong>textual factors.<br />

We may discern two salient features from the currents streams of research into sustainable Enterprise<br />

Architecture. First and foremost, there is a pr<strong>on</strong>ounced focus <strong>on</strong> artefact as a means to promote<br />

sustainability. This marks a sharp c<strong>on</strong>trast to other, more organizati<strong>on</strong>-based views such as Kay’s<br />

(1993) noti<strong>on</strong> of architecture as founded up<strong>on</strong> pers<strong>on</strong>al relati<strong>on</strong>ships, or Huisman’s (2003) view of<br />

architecture as a means to balance stakeholder demands and enterprise capabilities. Furthermore,<br />

change as such is not explored in any great detail but rather menti<strong>on</strong>ed in passing or vaguely alluded.<br />

Extant literature actively promotes agility and modularisati<strong>on</strong> as means to promote changeability, but<br />

does not give us any insight into whet nature of change itself. We wish to promote a deeper<br />

understanding of the complexities of change since Enterprise Architecture is not – and should not –<br />

be limited to tinkering with technology. At a bare minimum, we perceive that three aspects of change<br />

179


Thanos Magoulas et al.<br />

need to be made explicit: The area of effect, the deviati<strong>on</strong> from past/current practices, and the<br />

mindset with which we move forward.<br />

3. A three-dimensi<strong>on</strong>al framework for change<br />

In the following secti<strong>on</strong>, we will outline our framework <strong>on</strong>e dimensi<strong>on</strong> at a time. We do not assert that<br />

our classificati<strong>on</strong> is in any way definitive, but rather a serviceable starting point for a discourse into<br />

change in Enterprise Architecture.<br />

3.1 Perspectives <strong>on</strong> change<br />

The starting point of any change effort is a perceived dissatisfacti<strong>on</strong> between what is and what should<br />

be. Our preferences, skill-set and past experiences greatly influence how we perceive said<br />

dissatisfacti<strong>on</strong> and how we seek to resolve it (Langefors, 1973, p. 242-249). As sharing implicit<br />

knowledge is extremely difficult (Boisot, 1995, p. 72-73), these percepti<strong>on</strong>s are usually made explicit<br />

by means of various schools of thought, paradigms and methodologies in order to facilitate<br />

coordinated acti<strong>on</strong>. We shall limit ourselves to elucidating two extremes by which we may c<strong>on</strong>fr<strong>on</strong>t<br />

change efforts: Hard systems thinking and soft systems thinking. In additi<strong>on</strong> to listed references, we<br />

draw up<strong>on</strong> research by Argyris (1971), Tichy (1983), Bartunek & Moch (1987) and Ross, Weill &<br />

Roberts<strong>on</strong> (2006).<br />

Hard systems thinking is based up<strong>on</strong> operati<strong>on</strong>al research and as such essentially rooted in a<br />

positivist and reducti<strong>on</strong>ist world view. The underlying assumpti<strong>on</strong> is that each comp<strong>on</strong>ent of a system<br />

has a number of “absolute” properties that may be objectively ascertained. Building <strong>on</strong> this logical<br />

assumpti<strong>on</strong>, we may then also build a system that we deem desirable to the task at hand by using the<br />

proper comp<strong>on</strong>ents. We refer to this procedure as either “optimizing” or “goal-seeking” depending <strong>on</strong><br />

whether the system in questi<strong>on</strong> is mechanical or human (Checkland, 1985). Hence, when applied in<br />

practice, hard systems thinking tend to be based <strong>on</strong> deductive reas<strong>on</strong>ing. This puts the “problem<br />

solver” in the role of an external party to the change effort; perceiving it through codified variables<br />

such as a specificati<strong>on</strong> of requirements. Working in this manner, the organizati<strong>on</strong> utilizes planning and<br />

strategy as means to match the “system’s” functi<strong>on</strong>ality against requirements – present or predicted.<br />

Heavy emphasis is placed up<strong>on</strong> proper procedure such as normative methods or generic tools in<br />

order to safeguard desirable results (Mackenzie, 1984). Illustrating via healthcare, any change effort<br />

that is intended to improve any existing activity or routine may be characterized as a result of hard<br />

systems thinking.<br />

In c<strong>on</strong>trast, soft systems thinking is based up<strong>on</strong> <strong>on</strong> relativism and systems theory which states that<br />

the properties of a comp<strong>on</strong>ent are highly dependent <strong>on</strong> c<strong>on</strong>text and purpose. In other words, the<br />

value, utility, functi<strong>on</strong> et cetera of comp<strong>on</strong>ents should primarily be assessed by the efficacy with which<br />

they interc<strong>on</strong>nect to form a cohesive system (Churchman, 1971, p. 66-76). The distincti<strong>on</strong> between<br />

efficiency and efficacy is often important within the realm of social systems. Indeed, there may not be<br />

“problems” as such but rather a nebulous sense of dissatisfacti<strong>on</strong> or misalignment that has been<br />

present for a l<strong>on</strong>g time (Checkland, 2000). As such, the “problems” are not objectively ascertained but<br />

rather socially c<strong>on</strong>structed based <strong>on</strong> dialogue and discussi<strong>on</strong> am<strong>on</strong>g stakeholders. Given the<br />

subjective, even political nature of establishing a problem situati<strong>on</strong>, the “problem solver” is not an<br />

external party to the change effort, but rather an actor in elucidating and balancing the different<br />

perspectives. As soft systems thinking extends well bey<strong>on</strong>d technical systems, we must not <strong>on</strong>ly ask<br />

ourselves how to do something, but also what and why. Hence, before establishing any plan of acti<strong>on</strong><br />

(such as a strategy), we must first establish goals that motivate stakeholders to take acti<strong>on</strong> – or at the<br />

very least acquiesce. Only then may tangible steps be taken towards reducing the disparity between<br />

the present- and desired state of affairs. Over the past decade, healthcare has gradually shifted from<br />

being focused <strong>on</strong> the medical practiti<strong>on</strong>er to being focused <strong>on</strong> the patient. This necessitates a<br />

fundamental departure from existing values and mindsets as it directly affects the authority of the<br />

medical professi<strong>on</strong>al.<br />

3.2 Levels of change<br />

Our sec<strong>on</strong>d dimensi<strong>on</strong> depicts the scope of the change effort undertaken. A wider scope carries with<br />

it a wider and more diverse range of stakeholders. We must therefore balance patterns of authority<br />

and know-how in architectural modificati<strong>on</strong>s. In our framework, based up<strong>on</strong> research by Sim<strong>on</strong><br />

(1962), Ackoff (1967) and Armenakis & Bedeian (1999), we have chosen to distinguish between three<br />

different levels of change: Local-, structural- and inter-organizati<strong>on</strong>al change.<br />

180


Thanos Magoulas et al.<br />

Local changes are c<strong>on</strong>fined to a single organizati<strong>on</strong>al unit. In an industrial setting, this would be<br />

described as a technical subunit (Thomps<strong>on</strong>, 1967, p.10-11) whereas a more general descripti<strong>on</strong><br />

would be to describe it as an informati<strong>on</strong> domain (Magoulas & Pessi, 1998, p. 366-367). The most<br />

salient feature of local changes is the proximity between decisi<strong>on</strong>-maker and c<strong>on</strong>text. It is entirely<br />

possible that the impetus for change is an external factor, but the change effort is adapted to suit local<br />

c<strong>on</strong>diti<strong>on</strong>s. Using healthcare as a means to illustrate, any decisi<strong>on</strong> that is made at – and limited to – a<br />

single primary care unit would be c<strong>on</strong>sidered a local change.<br />

Structural changes affect several organizati<strong>on</strong>al units, creating a change effort that is wider in scope.<br />

We now c<strong>on</strong>cern ourselves with a managerial (Thomps<strong>on</strong>, 1967, p. 10-12) effort that impacts the<br />

wider informati<strong>on</strong> envir<strong>on</strong>ment (Magoulas & Pessi, 1998, p. 367-370). Kay (1993, p. 78-80) describes<br />

this as a change effort affecting the internal architecture of an enterprise. As there are now more than<br />

<strong>on</strong>e organizati<strong>on</strong>al unit involved, there is an inherent diversity in priorities and c<strong>on</strong>textual issues to<br />

address. Hence, there is a very real possibility that political issues impact structural changes<br />

(Davenport, Eccles & Prusak, 1992; Davenport & Prusak, 1997, p.67). A c<strong>on</strong>crete example from<br />

healthcare is the relatively recent (and <strong>on</strong>going) transiti<strong>on</strong> from organizati<strong>on</strong>al hierarchies to<br />

organizati<strong>on</strong>al processes. To some degree, this involves breaking with established structures and<br />

mindsets that have been instilled in generati<strong>on</strong>s of medical professi<strong>on</strong>als.<br />

Expanding our scope even further, we find ourselves face with inter-organizati<strong>on</strong>al changes. No<br />

l<strong>on</strong>ger limited to a single enterprise, these efforts c<strong>on</strong>cern the instituti<strong>on</strong>al level (Thomps<strong>on</strong>, 1967, p.<br />

10-12) of the organizati<strong>on</strong>, i.e. the societal c<strong>on</strong>tract by which the enterprise operates. Kay (1993, p.<br />

80-82) refers to this as the external architecture as it involves separate legal entities. There are<br />

several ways to exemplify inter-organizati<strong>on</strong>al changes based <strong>on</strong> healthcare. One relatively intuitive<br />

illustrati<strong>on</strong> would be the use of e-prescripti<strong>on</strong>s transmitted between healthcare providers and<br />

pharmacies. Another, perhaps more c<strong>on</strong>temporary, illustrati<strong>on</strong> would be the drive towards<br />

interoperability in healthcare between primary care, private practiti<strong>on</strong>ers, specialists and emergency<br />

care. While all of these facilities provide care to the same populous, they do so <strong>on</strong> different terms and<br />

with different purposes.<br />

3.3 Types of change<br />

In this secti<strong>on</strong> we define and exemplify enterprise changes based <strong>on</strong> their deviati<strong>on</strong> from existing<br />

structures and processes. We delineate these into three types of change: Incremental changes,<br />

transformati<strong>on</strong>al changes and reorientati<strong>on</strong>-based changes. This classificati<strong>on</strong> is based up<strong>on</strong><br />

research by Henders<strong>on</strong> & Clark (1990), Tushman & O’Reilly (1995) and Newman (2000).<br />

Incremental changes cover enterprise changes limited to improving existing structures using<br />

predominantly existing know-how. This is often referred to as single-loop learning (Argyris, 1977).<br />

These types of changes strive to promote efficiency in terms of quality, cost or time. This manner of<br />

gradual, c<strong>on</strong>trolled improvements has been widely adopted in the manufacturing industry under the<br />

guise of Total Quality <strong>Management</strong> (Oakland, 1993), Six Sigma (Brue, 2005) and Kaizen (Lee,<br />

Dugger & Chen, 1999). Within the c<strong>on</strong>text of healthcare, incremental changes take the form of<br />

c<strong>on</strong>tinuous improvement of care processes, tools or working schedules. While incremental changes<br />

may tend to be localized, this is not always the case. New legislati<strong>on</strong> may force healthcare providers<br />

to amend their practices <strong>on</strong> a nati<strong>on</strong>al scale – even if the change itself may be described as<br />

incremental and easily accommodated using existing organizati<strong>on</strong>al structures.<br />

Transformati<strong>on</strong>al changes are more far-reaching then aforementi<strong>on</strong>ed incremental ditto in that<br />

existing structures and/or processes must be altered in order to accommodate a new modus<br />

operandi. The ultimate goal is still unaltered, but the means by which <strong>on</strong>e strives to achieve this<br />

changes substantially. This is comm<strong>on</strong>ly referred to as sec<strong>on</strong>d-loop learning (Argyris, 1977). The<br />

ends of transformati<strong>on</strong>al changes are typically expressed in terms of effectiveness rather than<br />

efficiency, i.e. attaining the desired result rather than ec<strong>on</strong>omizing <strong>on</strong> resources. ICT has been treated<br />

as a powerful enabler of transformati<strong>on</strong>al changes due to the added ability to process vast quantities<br />

of data as exemplified by Business Process Reengineering (Hammer, 1990). Using healthcare as a<br />

backdrop, instructing the patient to administer his/her own care represents a transformati<strong>on</strong>al change.<br />

The fundamental goal of the caregiver is still the treatment (or cure) of the patient, but the patient has<br />

been given a greater degree of resp<strong>on</strong>sibility and aut<strong>on</strong>omy. In effect, the patient is now a coproducer<br />

of medical informati<strong>on</strong> rather than merely a factor thereof.<br />

181


Thanos Magoulas et al.<br />

An enterprise facing reorientati<strong>on</strong>-based change undergoes a fundamental change of its very identity.<br />

It is no l<strong>on</strong>ger a matter of doing things differently, but rather <strong>on</strong>e of doing different things. In other<br />

words, the ultimate goal or missi<strong>on</strong> of the enterprise is altered. In Soft systems Methodology, this is<br />

expressed as altering the root definiti<strong>on</strong>s (Checkland, 1981). It is unlikely that this profound manner of<br />

change is undertaken at any other level than top management, yet given the far-reaching implicati<strong>on</strong>s<br />

in terms of culture and core values, it is unlikely that such an undertaking is possible without support<br />

from all stakeholders.<br />

Within the c<strong>on</strong>text of healthcare, a reorientati<strong>on</strong> is most clearly exemplified by shifting <strong>on</strong>e’s focus<br />

from treatment of injury or disease to preventi<strong>on</strong> by means of acting pre-emptively to promote a<br />

healthy lifestyle.<br />

3.4 The three dimensi<strong>on</strong>s of change<br />

Having presented the dimensi<strong>on</strong>s individually, we now present the framework as we perceive it.<br />

Figure 1: A three-dimensi<strong>on</strong>al framework for change<br />

We illustrate the framework thusly in order to explicate two important issues related to architectural<br />

change efforts. First and foremost, there is no “maximum” level of complexity. We could expand our<br />

axes ad infinitum if we wanted to, but doing so would be of little use as the levels featured serve to<br />

cover most realistic situati<strong>on</strong>s. Sec<strong>on</strong>dly, increasing the scope of change, attempting a more radical<br />

departure from current practices or incorporating new knowledge are all potent challenges in their<br />

own right. What we wish to illustrate using this framework is that compiling these exacerbating factors<br />

is not tantamount to simply adding another variable – it really is an entirely new dimensi<strong>on</strong> of<br />

complexity.<br />

4. Discussi<strong>on</strong><br />

As we have outlined in the previous chapter, change is a complicated, multidimensi<strong>on</strong>al c<strong>on</strong>struct. As<br />

we move away from localized, incremental changes that are based up<strong>on</strong> pre-existing know-how, we<br />

find ourselves with a richer panthe<strong>on</strong> of stakeholders that must embrace new methodologies and new<br />

paradigms. A similar point has been made by Foster-Fishman, Nowell and Yang (2007) albeit from<br />

the perspective of community change efforts. Yet their discourse holds relevance as the chief<br />

“comp<strong>on</strong>ents” of enterprises – like communities – are the people that inhabit them. Another relevant<br />

similarity is that enterprises like communities tend to be diverse when viewed as a whole. It is unlikely<br />

that a large gathering of denizens or employees will display homogeneity of background, skill-set or<br />

experience. As it is the purpose of Enterprise Architecture to furnish the enterprise with a means to<br />

organize its c<strong>on</strong>stituent comp<strong>on</strong>ents, the architecture must be commensurate to the level of<br />

complexity experienced by the enterprise. This line of reas<strong>on</strong>ing is based up<strong>on</strong> Ashby’s law of<br />

requisite variety which states that a c<strong>on</strong>trol mechanism must allow for the same level as complexity as<br />

182


Thanos Magoulas et al.<br />

the system it is to c<strong>on</strong>trol (Waelchli, 1989). In our case, “c<strong>on</strong>trol mechanism” is perhaps a somewhat<br />

limited m<strong>on</strong>iker for EA, yet apt insofar as it shapes our understanding of the enterprise and as such<br />

directly influences our acti<strong>on</strong>s. In light of this, extant literature <strong>on</strong> change efforts in EA provides us<br />

with detailed perspectives <strong>on</strong> how to create flexibility in IT-artefacts, but says little with regards to the<br />

enterprise as such. Even less is said <strong>on</strong> the topic of change itself. Our interpretati<strong>on</strong> of Ashby’s law<br />

suggests that adopting a technical perspective when dealing with organizati<strong>on</strong>al issues is at the very<br />

least simplistic; an asserti<strong>on</strong> that is supported by IS research (Earl, 1993). The explicit need to<br />

address stakeholders in organizati<strong>on</strong>al change efforts have been highlighted by Hedberg (1980) as<br />

well as Davenport, Eccles and Prusak (1992). Kay (1993) has takes this perspective <strong>on</strong>e step further<br />

by asserting that stakeholders are the source of architecture, not a factor thereof.<br />

Expanding architectural design to include n<strong>on</strong>-technical aspects – i.e. stakeholders – adds several<br />

layers of complexity for the architect. Yet it is a necessary step if we are to design architectures that<br />

ameliorate problems rather than ignore them. Realizing the challenges brought <strong>on</strong> by complex change<br />

efforts, we should seek to design an architecture that allows the enterprise to absorb changes rather<br />

than go through frequent processes of redesign. Should a redesign be c<strong>on</strong>sidered necessary, then a<br />

fuller understanding of the complexities of change may guide us to design an architecture that<br />

enables us to keep change efforts as simple as possible; restricting as much as possible to localized,<br />

incremental changes that rely <strong>on</strong> tested and true methods. We believe that the framework presented<br />

in this paper is a first step towards a deeper understanding of sustainable change in Enterprise<br />

Architecture. A greater sense of sustainability allows us to bring about efficacious change that is<br />

suited to the enterprise envir<strong>on</strong>ment while c<strong>on</strong>currently promoting resource efficiency – be they<br />

temporal, pecuniary or intellectual in nature. This is the challenge that faces academia as well as<br />

practiti<strong>on</strong>ers and it is here that the full value of Enterprise Architecture can come to light.<br />

5. C<strong>on</strong>clusi<strong>on</strong>s<br />

In the course of this paper, we have attempted to ascertain how change efforts are treated in<br />

c<strong>on</strong>juncti<strong>on</strong> with Enterprise Architecture. Our purpose in this undertaking has been to further our<br />

understanding of how architectural changes may be imbued with a sense of sustainability that<br />

enables architectures to evolve based <strong>on</strong> the needs of the enterprise as well as the experiences of<br />

stakeholders.<br />

When studying change itself, it is readily apparent that change is not a simple phenomen<strong>on</strong> or act, but<br />

rather a complicated undertaking that cannot be reduced to a single scale or metric. Furthermore,<br />

there is a reciprocal relati<strong>on</strong>ship between architecture and change; we need architecture in order to<br />

effect purposeful change. Change, in turn, serves as an impetus for architectural evoluti<strong>on</strong>.<br />

Underestimating the inherent complexities by applying similar methods to all manner of change efforts<br />

will at best yield a capricious track record. Sustainable Enterprise Architecture can <strong>on</strong>ly be attained<br />

after we have enabled ourselves to better c<strong>on</strong>ceptualize and understand change. This is especially<br />

true when studying structures such as complex organizati<strong>on</strong>s operating in dynamic envir<strong>on</strong>ments. If<br />

we are to understand change, we require a c<strong>on</strong>struct with sufficient granularity so that we may<br />

ascertain how change efforts impact different avenues of the enterprise. We are then in a far better<br />

positi<strong>on</strong> to understand and discuss change am<strong>on</strong>g enterprise stakeholders with the aid of<br />

architectural design.<br />

In resp<strong>on</strong>se to the need for better understanding of sustainability in Enterprise Architecture, we<br />

propose a tentative three-dimensi<strong>on</strong>al framework for change. This framework furthers our<br />

understanding of enterprise change based <strong>on</strong> three essential dimensi<strong>on</strong>s. The relevance and<br />

usefulness of these three dimensi<strong>on</strong>s has been validated through seminars, workshops or advanced<br />

courses held with over 30 healthcare professi<strong>on</strong>als. While we feel that this is a step in the right<br />

directi<strong>on</strong>, there is still much work to be d<strong>on</strong>e in this area. We therefore call for a deeper discourse and<br />

further research into sustainability in Enterprise Architecture.<br />

References<br />

Ackoff, R. (1967). <strong>Management</strong> Misinformati<strong>on</strong> Systems. <strong>Management</strong> Science, Vol. 14 (4), 147-156.<br />

Aier, S (2004). Sustainability of Enterprise Architecture and EAI. The 2004 Internati<strong>on</strong>al Business Informati<strong>on</strong><br />

<strong>Management</strong> <str<strong>on</strong>g>C<strong>on</strong>ference</str<strong>on</strong>g> (Amman, Jordan) 2004.<br />

Aier, S. & Schoenherr, M. (2005). Sustainability of Enterprise Architecture and EAI – an empirical study. In<br />

Milutinovic, V. (Ed.) Proceedings of the Internati<strong>on</strong>al <str<strong>on</strong>g>C<strong>on</strong>ference</str<strong>on</strong>g> <strong>on</strong> Advances in Internet, Processing,<br />

Systems, and Interdisciplinary Research, IPSI-2005, MIT Cambridge/IPSI, Cambridge, Bost<strong>on</strong> MA.<br />

183


Thanos Magoulas et al.<br />

Argyris, C. (1971). <strong>Management</strong> Informati<strong>on</strong> Systems: The Challenge to Rati<strong>on</strong>ality and Emoti<strong>on</strong>ality.<br />

<strong>Management</strong> Science, Vol. 17 (6), 275-292.<br />

Argyris, C. (1977). Organizati<strong>on</strong>al Learning and <strong>Management</strong> Informati<strong>on</strong> Systems. Accounting, Organizati<strong>on</strong>s<br />

and Society, Vol. 2 (2), 113-123.<br />

Armenakis, A.A. & Bedeian, A.G. (1999). Organizati<strong>on</strong>al change: A review of theory and research in the 1990s.<br />

Journal of <strong>Management</strong>, Vol. 25 (3), 293-315.<br />

Bartunek, J.M. & Moch, M.K. (1987). First-order, sec<strong>on</strong>d-order, and third-order change and organizati<strong>on</strong>al<br />

development interventi<strong>on</strong>s: A cognitive approach. Journal of Applied Behavioral Science, Vol. 23 (4), 483-<br />

500.<br />

Boisot, M. H. (1995). Informati<strong>on</strong> Space. L<strong>on</strong>d<strong>on</strong>: Routledge.<br />

Brue, G. (2005). Six Sigma for Managers. New York: McGraw-Hill Professi<strong>on</strong>al Educati<strong>on</strong> Series.<br />

Checkland, P. (1981). Systems Thinking, Systems Practice. New York: John Wiley & S<strong>on</strong>s.<br />

Checkland, P. (1985). From Optimizing to Learning: A Development of Systems Thinking for the 1990s. Journal<br />

of Operati<strong>on</strong>al Research Soc. Vol. 36 (9), 757-767.<br />

Checkland, P. (2000). Soft Systems Methodology: A Thirty Year Retrospective. Systems Research and<br />

Behavioural Science, Vol.17, 11-58.<br />

Churchman, C. W. (1971). The Design of Inquiring Systems: Basic C<strong>on</strong>cepts of Systems and Organizati<strong>on</strong>. New<br />

York: Basic Books.<br />

Davenport, T.H., Eccles, R.J. & Prusak, L. (1992). Informati<strong>on</strong> Politics. Sloan management review, Fall 1992.<br />

Davenport, T.H.& Prusak, L. (1997). Informati<strong>on</strong> Ecology. New York: Oxford University Press.<br />

Earl, M.J. (1993). Experiences in Strategic Informati<strong>on</strong> Systems Planning. MIS Quarterly, Vol. 17 (1), 1-24.<br />

Fischer, R., Aier, S. & Winter, R. (2007). A Federated Approach to Enterprise Architecture Model Maintenance.<br />

Enterprise Modelling and Informati<strong>on</strong> Systems Architectures, Vol. 2 (2), 14–22.<br />

Foster-Fishman, P.G., Nowell, B. & Yang. (2007). Putting the system back into systems change: A framework for<br />

understanding and changing organizati<strong>on</strong>al and community systems. American Journal of Community<br />

Psychology, Vol. 39 (3-4), 197-215.<br />

IFIP-IFAC Task force <strong>on</strong> Architectures for Enterprise Integrati<strong>on</strong>. (2003). The Generalised Enterprise Reference<br />

Architecture and Methodology: Versi<strong>on</strong> 1.6.3 (Final). In Bernus, P., Nemes, L. and Schmidt, G. (Eds.)<br />

Handbook <strong>on</strong> Enterprise Architecture. Berlin: Springer-Verlag.<br />

Gardner, D. (2012). Enterprise architecture and enterprise transformati<strong>on</strong>: Related but distinct c<strong>on</strong>cepts that can<br />

change the world. Zdnet, Febuary 22, 2012. http://www.zdnet.com/blog/gardner/enterprise-architecture-andenterprise-transformati<strong>on</strong>-related-but-distinct-c<strong>on</strong>cepts-that-can-change-the-world/4530<br />

Hammer, M. (1990). Re-engineering Work: D<strong>on</strong>'t Automate, Obliterate. Harvard Business Review, July-August,<br />

104-112.<br />

Hedberg, B. (1980). Using computerized Informati<strong>on</strong> Systems to design better organizati<strong>on</strong> and jobs. In Bjørn-<br />

Andersen, N. (Ed.) The Human Side of Informati<strong>on</strong> Processing. Amsterdam: North-Holland.<br />

Henders<strong>on</strong>, R. M. & Clark, K.B. (1990). Architectural Innovati<strong>on</strong>: The Rec<strong>on</strong>figurati<strong>on</strong> of Existing Product<br />

Technologies and the Failure of Established firms. Administrative Science Quarterly, Vol. 35 (1), 9-30.<br />

Huisman, B. (2003) Informati<strong>on</strong> Strategy: An Introducti<strong>on</strong>. In Berg, M. (Ed.) Health Informati<strong>on</strong> <strong>Management</strong>,<br />

Integrating IT in health care work. L<strong>on</strong>d<strong>on</strong>: Routledge.<br />

Kay, J. (1993). Foundati<strong>on</strong>s of Corporate Success. Oxford: Oxford University Press.<br />

Kluge, C., Dietzsch, A., & Rosemann, M. (2006). Fostering an Enterprise Architecture’s Value Propositi<strong>on</strong> Using<br />

Dedicated Presentati<strong>on</strong>s Strategies. Proceedings of the CAISE '06 Workshop <strong>on</strong> Business/IT Alignment and<br />

Interoperability (BUSITAL '06), Luxemburg, 2006.<br />

Kluge, C., Dietzsch, A.& Rosemann, M. (2006) How to realise corporate value from enterprise architecture. ECIS<br />

2006 Proceedings. Paper 133.<br />

Langefors, B. (1973). Theoretical Analysis of Informati<strong>on</strong> Systems. 4th Editi<strong>on</strong>. Lund: Studentlitteratur.<br />

Langefors, B. (1975). C<strong>on</strong>trol Structrure and Formalized Informati<strong>on</strong> Analysis in Organizati<strong>on</strong>s. In Grochla, E. &<br />

Szyperski, N. (Eds.) Informati<strong>on</strong> Systems and Organizati<strong>on</strong>al Structure. Berlin and New York: Walter de<br />

Gruyter.<br />

Lee, S. S., Dugger, J. C. & Chen, J. C. (2000). Kaizen: An Essential tool for Inclusi<strong>on</strong> in Industrial Technology<br />

Curricula, Journal of Industrial Technology, Vol. 16 (1), 1-7.<br />

McKenzie, K.D. (1984). A Strategy and desiderata for organizati<strong>on</strong>al design. Human Systems <strong>Management</strong>, Vol.<br />

4, 201-213.<br />

Magoulas, T. & Pessi, K. (1998). Strategic IT-management. (Doctoral Thesis) Gothenburg: Gothenburg<br />

University, Department of Informatics.<br />

Magoulas, T., Hadzic, A., Saarikko, T. & Pessi, K. (2012). Alignment in Enterprise Architecture: A Comparative<br />

Analysis of Four Architectural Approaches. Electr<strong>on</strong>ic Journal of Informati<strong>on</strong> Systems Evaluati<strong>on</strong>. Vol. 15<br />

(1), 88-101.<br />

Nadler, D. & Tushman, M. (1995). Types of organizati<strong>on</strong>al change: From incremental improvement to<br />

disc<strong>on</strong>tinuous transformati<strong>on</strong>. In Nadler, D.A., Shaw, R.B. & Walt<strong>on</strong>, A.E. (Eds.) Disc<strong>on</strong>tinuous Change:<br />

Leading Organizati<strong>on</strong>al Transformati<strong>on</strong>. San Fransisco: Jossey-Bass.<br />

Newman, K. L. (2000). Organizati<strong>on</strong>al transformati<strong>on</strong> during instituti<strong>on</strong>al upheaval. Academy of <strong>Management</strong><br />

Review, Vol. 25 (3), 602–619.<br />

Orlikowski, W. (1993). Case Tools as Organizati<strong>on</strong>al Change: Investigating Incremental and Radical Changes in<br />

Systems Development. MIS Quarterly, Vol. 17 (3), 309-340.<br />

184


Thanos Magoulas et al.<br />

Oakland, J. (1993). Total Quality <strong>Management</strong>: The Route for Improving Performance. Oxford: Butterworth<br />

Heinemann.<br />

Ren, M. & Lyytinen, K. (2008). Building Enterprise Architecture Agility and Sustenance with SOA.<br />

Communicati<strong>on</strong>s of the Associati<strong>on</strong> for Informati<strong>on</strong> Systems, Vol. 22 (1), 75-86.<br />

Ross, J., Weill, P. & Roberts<strong>on</strong>, D. (2006). Enterprise Architecture as Strategy. Bost<strong>on</strong>: Harvard University Press.<br />

Schelp, J. Aier, S. (2009). SOA and EA—Sustainable C<strong>on</strong>tributi<strong>on</strong>s for increasing Corporate Agility. Proceedings<br />

of HICSS-42. IEEE Computer Society, Los Alamitos (2009).<br />

Sim<strong>on</strong>, H. (1962). The Architecture of Complexity. Proceedings of the American Philosophical Society, Vol. 106<br />

(6), 467-482.<br />

Sowa, J.F. & Zachman, J.A. (1992). Extending and Formalizing the Framework for Informati<strong>on</strong> Systems<br />

Architecture. IBM Systems Journal 31 (3), 590-616.<br />

Thomps<strong>on</strong>, J. D. (1967). Organizati<strong>on</strong>s in Acti<strong>on</strong>. New York: McGraw-Hill.<br />

Tichy, N.M. (1983). Managing Strategic Change – Technical, Political, and Cultural Dynamics. Chichester: John<br />

Wiley & S<strong>on</strong>s.<br />

Tushman, M.L. & O’Reilly, C.A. (1996). Ambidextrous organizati<strong>on</strong>s: Managing evoluti<strong>on</strong>ary and revoluti<strong>on</strong>ary<br />

change. California <strong>Management</strong> Review, 38 (4), 8-30.<br />

Waelchli, F. (1989). The VSM and Ashby’s Law as illuminants of historical management thought. In Espejo, R. &<br />

Harnden, R. (Eds.) The Viable System Model: Interpretati<strong>on</strong>s and Interpretati<strong>on</strong>s of Stafford Beer’s VSM.<br />

Chichester: John Wiley & S<strong>on</strong>s.<br />

Weick, K.E. & Quinn, R.E. (1999). Organizati<strong>on</strong>al change and development. Annual review of psychology, Vol.<br />

50, 361-386.<br />

185


Applying Structural Equati<strong>on</strong> Modelling to Exploring the<br />

Relati<strong>on</strong>ship Between Organisati<strong>on</strong>al Trust and Quality of<br />

Work Life<br />

Nico Martins and Yolandi van der Berg<br />

University of South Africa, Pretoria, South Africa<br />

martin@unisa.ac.za<br />

Yolandi.VanDerBerg@za.sabmiller.com<br />

Abstract: Dissatisfacti<strong>on</strong> with working life is a problem affecting almost all employees during their working<br />

career, regardless of positi<strong>on</strong> or status. Although many managers seek to reduce job dissatisfacti<strong>on</strong> at all<br />

organisati<strong>on</strong>al levels, they sometimes find it difficult to isolate and identify all of the attributes, which affect and<br />

influence the quality of working life. Some researchers proclaim that the success of Quality of Work Life<br />

programmes will depend <strong>on</strong> the ability of the organisati<strong>on</strong> to reinforce high levels of trust. Quality of work life is<br />

assumed to affect various organisati<strong>on</strong>al factors such as job effort and performance, organisati<strong>on</strong>al identificati<strong>on</strong>,<br />

job satisfacti<strong>on</strong> and job involvement. The aim of this quantitative research, based <strong>on</strong> theoretical and empirical<br />

research, is to determine the relati<strong>on</strong>ship between organisati<strong>on</strong>al trust and quality of work life. A validated<br />

organisati<strong>on</strong>al trust questi<strong>on</strong>naire (c<strong>on</strong>sisting of Big Five pers<strong>on</strong>ality c<strong>on</strong>structs, managerial practices and<br />

organisati<strong>on</strong>al trust dimensi<strong>on</strong>s) and a quality of work life questi<strong>on</strong>naire (11 dimensi<strong>on</strong>s) were used in the<br />

research. Two hundred sales representatives of a marketing company, participated in the research. An internetbased<br />

survey methodology was used to collect primary data from a probability sample of 282 sales<br />

representatives’ resp<strong>on</strong>dents with a 72% resp<strong>on</strong>se rate. Resp<strong>on</strong>ses were analysed using quantitative techniques<br />

and Structural Equati<strong>on</strong> Modelling. Results c<strong>on</strong>firm a positive relati<strong>on</strong>ship between the Managerial Practices and<br />

Organisati<strong>on</strong>al Trust, and a lower relati<strong>on</strong>ship between the Pers<strong>on</strong>ality dimensi<strong>on</strong>s and Organisati<strong>on</strong>al Trust.<br />

With regard to the Quality of Work Life, a positive relati<strong>on</strong>ship was noted with Managerial Practices but again<br />

lower relati<strong>on</strong>ship with the Pers<strong>on</strong>ality c<strong>on</strong>structs. The study strengthened and focused attenti<strong>on</strong> <strong>on</strong> the<br />

importance of building good trust relati<strong>on</strong>ships within an organisati<strong>on</strong>, as it seems as though the Pers<strong>on</strong>ality traits<br />

and Managerial Practices of managers will not <strong>on</strong>ly influence the trust relati<strong>on</strong>ship experienced by employees,<br />

but also their experience of a Quality of Work Life.<br />

Keywords: quality of work life (QWL), organisati<strong>on</strong>al trust, structural equati<strong>on</strong> modelling<br />

1. Introducti<strong>on</strong><br />

The extent and rate of change within organisati<strong>on</strong>s has created renewed interest in the quality of<br />

employees’ work lives, particularly in South Africa where organisati<strong>on</strong>s have to deal with cultural<br />

diversity, the ethnic compositi<strong>on</strong> of the workforce, and changes in value systems and beliefs (Kirby<br />

and Harter, 2001; Kotzé, 2005; Sekwena, 2007). Although many managers seek to reduce job<br />

dissatisfacti<strong>on</strong> at all organisati<strong>on</strong>al levels, including their own, they sometimes find it difficult to isolate<br />

and identify all of the factors which affect and influence the QWL (Huang, Lawler and Lei, 2007; May,<br />

Lau and Johns<strong>on</strong>, 1999; Walt<strong>on</strong>, 1973).<br />

According to Kaushik and T<strong>on</strong>k (2008) an employee’s QWL is determined by the interacti<strong>on</strong> of<br />

pers<strong>on</strong>al and situati<strong>on</strong>al factors involving both pers<strong>on</strong>al (subjective) and external (objective) aspects<br />

of work-related rewards and experiences. According to Kotzé (2005), the changes in the ethnic<br />

compositi<strong>on</strong> of the South African workforce, specifically with regard to changes in beliefs and value<br />

systems, as well as the greater importance placed <strong>on</strong> knowledgeable workers, are factors which may<br />

influence QWL. Affirmed and emphasised by Martins (2000) and Schoorman, Mayer and Davis<br />

(2007), these changes in the workforce may also lead to an increase in the importance of trust in<br />

organisati<strong>on</strong>s. QWL is assumed to affect various organisati<strong>on</strong>al factors (job effort and performance,<br />

organisati<strong>on</strong>al identificati<strong>on</strong>, job satisfacti<strong>on</strong> and job involvement) (Ballou and Godwin, 2007), while<br />

Organisati<strong>on</strong>al Trust is the employee’s expectati<strong>on</strong> of the reliability of the organisati<strong>on</strong>’s promises and<br />

acti<strong>on</strong>s (Carmeli, 2005; Politis, 2003). L<strong>on</strong>g, Sitkin and Cardinal (2003) urge managers to build trust<br />

between employees and the organisati<strong>on</strong> in order to enhance organisati<strong>on</strong>al effectiveness. Martins<br />

and V<strong>on</strong> der Ohe (2002) also indicate that trust is created by leadership, which in turn influences<br />

relati<strong>on</strong>ships and job satisfacti<strong>on</strong>.<br />

Research has further shown that QWL is not <strong>on</strong>ly a significant determinant of various enviable<br />

organisati<strong>on</strong>al outcomes, but that it also significantly influences the n<strong>on</strong>working life of an individual<br />

and is an important predictor of the life satisfacti<strong>on</strong>, health and psychological wellbeing of employees<br />

(Ballou and Godwin, 2007; Kaushik and T<strong>on</strong>k, 2008; 2010; Martel and Dupuis, 2006; Sirgy, Efraty,<br />

186


Nico Martins and Yolandi van der Berg<br />

Siegel and Lee, 2001; Srivastava, 2008; Wils<strong>on</strong>, DeJoy, Vandenberg, Richards<strong>on</strong> and McGrath,<br />

2004; Wright and B<strong>on</strong>ett, 2007).<br />

2. Organisati<strong>on</strong>al trust<br />

Trust can be regarded as a multidimensi<strong>on</strong>al c<strong>on</strong>struct, c<strong>on</strong>sisting of a cognitive (belief about<br />

another’s trustworthiness), affective (role of emoti<strong>on</strong>s in the trust process) and behavioural (relying <strong>on</strong><br />

another and disclosing sensitive informati<strong>on</strong>) base (Büssing, 2002; Gillespie and Mann, 2004;<br />

Rousseau, Sitkin, Burt and Camerer, 1998; Schoorman et al, 2007). Tschannen-Moran and Hoy<br />

(2000, p 556) c<strong>on</strong>sequently proposed a multidimensi<strong>on</strong>al definiti<strong>on</strong> of trust, namely: “Trust is <strong>on</strong>e<br />

party’s willingness to be vulnerable to another party based <strong>on</strong> the c<strong>on</strong>fidence that the latter party is (a)<br />

benevolent, (b) reliable, (c) competent, (d) h<strong>on</strong>est, and (e) open.”<br />

Despite the differences in c<strong>on</strong>ceptualisati<strong>on</strong>, there are a number of comm<strong>on</strong> elements unifying the<br />

many different definiti<strong>on</strong>s of trust. In particular there seems to be an agreement that trust is “the<br />

willingness to be vulnerable based <strong>on</strong> the positive expectati<strong>on</strong>s of the intenti<strong>on</strong>s or behaviour of<br />

others” (Mayer, Davis and Schoorman, 1995, p 712). Sec<strong>on</strong>dly, it seems that interdependence and<br />

uncertainty are necessary c<strong>on</strong>diti<strong>on</strong>s for trust to develop. In line with the above and taking into<br />

account that this research study is d<strong>on</strong>e within an organisati<strong>on</strong>al c<strong>on</strong>text, the author uses the<br />

definiti<strong>on</strong> provided by V<strong>on</strong> der Ohe, Martins and Roode (2004, p 6). Organisati<strong>on</strong>al Trust is therefore<br />

defined as “the choice to make <strong>on</strong>eself vulnerable with the express belief in the positive intent and<br />

commitment to the mutual gain of all parties involved in the relati<strong>on</strong>ship”.<br />

According to Hay (2002) and Lämsä and Pučėtaitė (2006), the importance of trust in organisati<strong>on</strong>s is<br />

likely to increase over the next few years. This is reiterated by Bews and Rossouw (2002) and Martins<br />

(2000), specifically in relati<strong>on</strong> to South Africa, due to the changing compositi<strong>on</strong> of the workforce and<br />

the focus <strong>on</strong> employment equity. In the development of a trust model, Martins (2000) and Martins and<br />

V<strong>on</strong> der Ohe (2002) also identified the Big Five Pers<strong>on</strong>ality aspects as significant indicators of trust,<br />

and their results provided support for the claim that Pers<strong>on</strong>ality characteristics, together with<br />

Managerial Practices – informati<strong>on</strong>-sharing, work support, credibility and team management – have<br />

an influence <strong>on</strong> the trust relati<strong>on</strong>ships between managers and employees. This overview links to the<br />

noti<strong>on</strong> that Organisati<strong>on</strong>al Trust is not necessarily an interpers<strong>on</strong>al form of trust, but rather a systems<br />

form of trust deriving from structures and processes within an organisati<strong>on</strong>, such as fairness and<br />

perceived organisati<strong>on</strong>al support (Bagraim and Hime, 2007), which in turn relates to the QWL an<br />

employee experiences within the organisati<strong>on</strong>.<br />

3. Quality of work life<br />

Most individuals spend a great deal of their time participating in job- or work-related activities and<br />

even plan their time, living standards and social interacti<strong>on</strong> around the demands of their work, and to<br />

a large extent people define themselves and others in terms of their work, making QWL in<br />

organisati<strong>on</strong>s a major comp<strong>on</strong>ent of quality of life in general (Kotzé, 2005; Rathi, 2010). Although<br />

QWL is a term used today in almost every area of organisati<strong>on</strong>al activity, definiti<strong>on</strong>s of QWL tend to<br />

change focus c<strong>on</strong>tinually and it has been viewed in various ways: as a movement, a set of<br />

organisati<strong>on</strong>al interventi<strong>on</strong>s (approaches to management in organisati<strong>on</strong>s) and as a type of working<br />

life experienced by employees (reflecting the affective evaluati<strong>on</strong> of individuals) ( Wyatt and Wah,<br />

2001). Extensive research <strong>on</strong> the definiti<strong>on</strong> and measurement of QWL from a range of disciplines has<br />

emerged, since it was first introduced in the 1950’s, and Hannif et al (2008, p 274) suggest that three<br />

categories of definiti<strong>on</strong> are found in the literature: (i) a c<strong>on</strong>cept c<strong>on</strong>cerned with employees’ job<br />

satisfacti<strong>on</strong>; (ii) a c<strong>on</strong>cept going bey<strong>on</strong>d job satisfacti<strong>on</strong> and encompassing subjective wellbeing; and<br />

(iii) a “dynamic, multidimensi<strong>on</strong>al c<strong>on</strong>struct that incorporates any number of measures – objective and<br />

subjective – relating to employment quality”.<br />

Schneider and Dachler (1978) in Kaushik and T<strong>on</strong>k (2008, p 36) found the feelings employees have<br />

about their job “tend to be stable over time and might be a product of specific pers<strong>on</strong>ality traits”. As<br />

already menti<strong>on</strong>ed, Pers<strong>on</strong>ality traits are psychological in nature, relatively stable over time, and<br />

provide the reas<strong>on</strong>s for behaviour (Church, 2000); they seem to be interrelated with trust and QWL by<br />

means of the Big Five Pers<strong>on</strong>ality factors. Research from various sources has found a link between<br />

the Big Five Pers<strong>on</strong>ality factors and dimensi<strong>on</strong>s relating to QWL such as job performance<br />

(Bozi<strong>on</strong>elos, 2004; Gellatly and Irving, 2001; Hurtz and D<strong>on</strong>ovan, 2000; Rothmann and Coetzer,<br />

2003), job satisfacti<strong>on</strong> (Goodstein and Lany<strong>on</strong>, 1999; Judge, Higgins and Cable, 2000; Thoresen,<br />

Kaplan, Barsky, Warren and De Cherm<strong>on</strong>t, 2003), emoti<strong>on</strong>al intelligence (Salgado, 2002),<br />

187


Nico Martins and Yolandi van der Berg<br />

organisati<strong>on</strong>al engagement (Bozi<strong>on</strong>elos, 2004), job proficiency (Salgado, 2002), organisati<strong>on</strong>al<br />

commitment (Thoresen et al, 2003), work and time pressures (Dijkstra and Fred, 2005; Morgan and<br />

de Bruin, 2010; Pienaar, Rothmann and Van de Vijver, 2007) work-life balance (Thoms<strong>on</strong> and de<br />

Bruin, 2007; Wayne, Musisca and Flees<strong>on</strong>, 2004) and reacti<strong>on</strong> to change (Vacola, Tsaousis and<br />

Nikolaou, 2004). Research c<strong>on</strong>ducted by Kaushik and T<strong>on</strong>k (2008) found a positive correlati<strong>on</strong><br />

between the c<strong>on</strong>struct QWL and three of the Big Five dimensi<strong>on</strong>s of Pers<strong>on</strong>ality, namely extraversi<strong>on</strong>,<br />

agreeableness and c<strong>on</strong>scientiousness. In additi<strong>on</strong>, research by Rothmann and Coetzer (2003)<br />

indicated that Pers<strong>on</strong>ality dimensi<strong>on</strong>s were related to management performance and identified<br />

emoti<strong>on</strong>al stability, resourcefulness and agreeableness as being significantly related to management<br />

performance. Shaw (2005, p 249) proposes that the success of QWL programmes will depend <strong>on</strong> the<br />

ability of the organisati<strong>on</strong> to “reinforce high levels of trust”, which in turn will improve organisati<strong>on</strong>al<br />

performance.<br />

Apart from its positive relati<strong>on</strong>ships with various dimensi<strong>on</strong>s of the QWL c<strong>on</strong>struct, as well as findings<br />

directly relating it to the c<strong>on</strong>struct (Kaushik and T<strong>on</strong>k, 2008), the Big Five Pers<strong>on</strong>ality aspects are also<br />

significant indicators of trust (V<strong>on</strong> der Ohe and Martins, 2010). However, it does seem as if there is a<br />

lack of research into the relati<strong>on</strong>ship between QWL and Organisati<strong>on</strong>al Trust.<br />

From the above the following hypotheses are formulated:<br />

Hypothesis 1: There is a positive relati<strong>on</strong>ship between organisati<strong>on</strong>al trust (The Big Five and<br />

managerial practices dimensi<strong>on</strong>s) and trust.<br />

Hypothesis 2: There is a positive relati<strong>on</strong>ship between quality of work life dimensi<strong>on</strong>s and<br />

organisati<strong>on</strong>al trust (The Big Five and managerial practices dimensi<strong>on</strong>s).<br />

4. Research methodology<br />

4.1 Research approach<br />

Structural Equati<strong>on</strong> Modelling (SEM) and correlati<strong>on</strong> analysis were used to test the relati<strong>on</strong>ship<br />

between the various factors or dimensi<strong>on</strong>s of Organisati<strong>on</strong>al Trust and QWL.<br />

4.2 Research method<br />

4.2.1 Research participants<br />

All 282 sales representatives across an organisati<strong>on</strong> were invited to participate in the research. In<br />

total, 203 participants completed the <strong>on</strong>line questi<strong>on</strong>naire (72% resp<strong>on</strong>se rate).Of these, 133 were<br />

male (65.5%) and 70 (34.5%) were female. The majority of resp<strong>on</strong>dents were African (124 or 61.1%),<br />

below the age of 46 years (175 or 86%), and had a tenure of two to five years (80 or 39.4%).<br />

4.2.2 Measuring instruments<br />

A combined Organisati<strong>on</strong>al Trust and QWL questi<strong>on</strong>naire c<strong>on</strong>sisting of six biographical questi<strong>on</strong>s, 92<br />

Organisati<strong>on</strong>al Trust questi<strong>on</strong>s and 59 QWL questi<strong>on</strong>s was posted <strong>on</strong> a survey company’s website<br />

with an open invitati<strong>on</strong> for sales employees to participate. To measure Organisati<strong>on</strong>al Trust the Trust<br />

audit survey was used (Martins, 2000), the questi<strong>on</strong>naire. Secti<strong>on</strong> five encompassed the QWL<br />

c<strong>on</strong>struct and was measured by means of the Leiden Quality of Work Questi<strong>on</strong>naire (Van der Doef<br />

and Maes, 1999 and 2002).<br />

5. Results<br />

Inferential statistics included the Cr<strong>on</strong>bach’s alpha and c<strong>on</strong>firmatory factor analysis (CFA) to c<strong>on</strong>firm<br />

the reliability of the instruments. SEM multivariate analysis technique was used to determine the<br />

relati<strong>on</strong>ship between the c<strong>on</strong>structs (Organisati<strong>on</strong>al Trust and QWL) and the independent dimensi<strong>on</strong><br />

of trust to test the theoretical model. CFA, path analysis and regressi<strong>on</strong> analysis within SEM were<br />

used to test the two hypotheses. Two SEM approaches were subsequently followed, namely the<br />

strictly c<strong>on</strong>firmatory approach (to c<strong>on</strong>firm a structural model specified by another researcher) and the<br />

model development approach (to find models into which the data fitted well statistically) (Gars<strong>on</strong>,<br />

2004; Schumacker and Lomax, 2004).<br />

188


5.1 Reliability analysis<br />

Nico Martins and Yolandi van der Berg<br />

The Cr<strong>on</strong>bach’s alpha was used to determine the internal reliability of items within each factor (see<br />

Table 1). An acceptable value for Cr<strong>on</strong>bach’s alpha is between 0.70 and 0.80 and values substantially<br />

lower indicate an unreliable scale (Field, 2005). For the purpose of this research study, a reliability<br />

coefficient of 0.70 or higher was c<strong>on</strong>sidered an acceptable score of internal c<strong>on</strong>sistency. Based <strong>on</strong><br />

each factor’s Cr<strong>on</strong>bach’s alpha, it was determined that all factors included within the Organisati<strong>on</strong>al<br />

Trust dimensi<strong>on</strong> had a str<strong>on</strong>g internal reliability with the lowest score being 0.602 obtained for<br />

Informati<strong>on</strong> Sharing. The reliability coefficient of the factors, which forms part of the QWL dimensi<strong>on</strong>,<br />

appears to vary between -0.179 and 0.908 with five of these reliability coefficients being above 0.9 –<br />

which can be regarded as acceptable internal c<strong>on</strong>sistency (Kline, 1999). The item analysis based <strong>on</strong><br />

Cr<strong>on</strong>bach’s alpha suggests there was a negative relati<strong>on</strong>ship between some items, that is, decisi<strong>on</strong><br />

authority and job insecurity, after recoding took place. Based <strong>on</strong> these reliability results, informati<strong>on</strong>sharing<br />

(0.602), and decisi<strong>on</strong> authority was excluded from SEM Model due to the weak Cr<strong>on</strong>bach<br />

alpha’s. There is, however, no obvious reas<strong>on</strong> for the negative Cr<strong>on</strong>bach’s alpha obtained for job<br />

insecurity, as there did not appear to be any coding error and it was therefore decided to include it as<br />

part of the model. Overall, it can be c<strong>on</strong>cluded that the internal c<strong>on</strong>sistency (reliability) of the overall<br />

Organisati<strong>on</strong>al Trust questi<strong>on</strong>naire and the factors are c<strong>on</strong>sistent in what it is intended to measure.<br />

With regard to the Leiden Quality of Work Questi<strong>on</strong>naire and its factors, internal reliability seems to<br />

vary between the various factors and can definitely be improved.<br />

Table 1: Results of reliability analysis<br />

Pers<strong>on</strong>ality<br />

Managerial Practices<br />

Dimensi<strong>on</strong><br />

Cr<strong>on</strong>bach<br />

Alpha<br />

N of<br />

items Comments<br />

Organisati<strong>on</strong>al Trust<br />

C<strong>on</strong>scientiousness 0.954 8<br />

Extraversi<strong>on</strong> 0.940 7<br />

Agreeableness 0.980 8<br />

Emoti<strong>on</strong>al Stability 0.952 5<br />

Resourcefulness 0.852 7<br />

Trust Relati<strong>on</strong>ship 0.941 5<br />

Credibility 0.944 15<br />

Work Support 0.945 4<br />

Informati<strong>on</strong> Sharing 0.602 4 Not included in SEM<br />

Team <strong>Management</strong> 0.947 8<br />

Change which has occurred 0.940 11<br />

Interpers<strong>on</strong>al Trust 0.874 9<br />

Quality of Work Life<br />

Skill Discreti<strong>on</strong> 0.598 8<br />

Decisi<strong>on</strong> Authority -0.179 4<br />

Task C<strong>on</strong>trol 0.536 4<br />

Work and Time Pressure 0.354 3<br />

Role Ambiguity 0.811 6<br />

Physical Exerti<strong>on</strong> 0.596 3<br />

Hazardous Exposure 0.852 8<br />

Not included in SEM<br />

Job Insecurity -0.125 3 Not included in SEM<br />

Lack of Meaningfulness 0.613 3<br />

Social Support Supervisor 0.888 6<br />

Social Support Colleagues 0.908 11<br />

Job Satisfacti<strong>on</strong> 0.843 5 One negative questi<strong>on</strong> was recoded<br />

189


5.2 SEM results<br />

Nico Martins and Yolandi van der Berg<br />

Alternative models were tested <strong>on</strong> the basis of the theory and changes to the structural and/or<br />

measurement models were made as suggested by the SEM modificati<strong>on</strong> indices. In figure 1, the<br />

relati<strong>on</strong>ship between the c<strong>on</strong>structs Organisati<strong>on</strong>al Trust and QWL is depicted. The path diagram and<br />

parameter estimates are illustrated. Results revealed a n<strong>on</strong>-significant chi-square = 622.252 based <strong>on</strong><br />

0.196 degrees of freedom with a probability of 0.000. The ratio of chi-square and degrees of freedom<br />

(x²/df) was equal to 3.175 indicating an adequate fit (a value of between 2 and 5 is believed to be a<br />

good fit) (Bollen and L<strong>on</strong>g, 1993). C<strong>on</strong>tradictory to this an RMSEA value of 0.104 was obtained.<br />

According to Gars<strong>on</strong> (2004) an RMSEA value of 0.05 or less indicates a close approximati<strong>on</strong> and<br />

values of up to 0.08 suggests a reas<strong>on</strong>able fit of the model in the populati<strong>on</strong>. A value of 0.104<br />

therefore suggests a moderate fit within the populati<strong>on</strong>.<br />

Figure 1: Relati<strong>on</strong>ship between organisati<strong>on</strong>al trust and quality of work life<br />

190


Nico Martins and Yolandi van der Berg<br />

The GFI is 0.754, which also indicates a moderate fit. In additi<strong>on</strong>, the CFI equals 0.910, reflecting a<br />

good fit. The NFI equals 0.875 and the NNFI equals 0.894, both reflecting an adequate fit. Based <strong>on</strong><br />

the above, it is therefore believed the structural model, based <strong>on</strong> these indices, achieved a moderate<br />

fit. Analysing the SEM correlati<strong>on</strong> coefficients between the various variables (see table 2), the model<br />

indicates moderate correlati<strong>on</strong>s between QWL and Managerial Practices (0.68) as well as between<br />

QWL and Pers<strong>on</strong>ality aspects (0.54).<br />

The Pears<strong>on</strong> product-moment correlati<strong>on</strong> coefficient was furthermore used to calculate the<br />

correlati<strong>on</strong>s between Organisati<strong>on</strong>al Trust, QWL, Pers<strong>on</strong>ality and Managerial Practices (see table 3).<br />

All correlati<strong>on</strong> coefficients were significant at the 0.01 level (2-tailed).<br />

Table 2: SEM correlati<strong>on</strong>s coefficients between organisati<strong>on</strong>al trust and QWL<br />

Dimensi<strong>on</strong> Correlati<strong>on</strong> SE P<br />

QWL Pers<strong>on</strong>ality 0.541 0.052 ***<br />

QWL Managerial Practices 0.679 0.039 ***<br />

Pers<strong>on</strong>ality Managerial Practices 0.790 0.029 ***<br />

e 22 e 23 0.475 0.055 ***<br />

e 6 e 7 0.488 0.049 ***<br />

e 4 e 3 0.453 0.056 ***<br />

e 5 e 9 0.363 0.063 ***<br />

e 7 e 8 0.358 0.052 ***<br />

e 13 e 14 0.337 0.069 ***<br />

e 5<br />

SE = Standard error<br />

e10 -0.316 0.085 ***<br />

P = probability value (


Nico Martins and Yolandi van der Berg<br />

explaining 95.3% of the variance, and team management explained 91.5% of the variance with an<br />

estimate of 7.47. Change which has occurred (estimate of 4.98) and interpers<strong>on</strong>al trust (estimate of<br />

5.18) seem to have the lowest impact <strong>on</strong> trust, explaining 16% and 41.7% or the variance<br />

respectively. Within the QWL dimensi<strong>on</strong>, social support from colleagues has the highest impact<br />

(estimate of 6.75) explaining 97.5% of the variance, followed by social support from the supervisor<br />

with an estimate of 4.01, explaining 93.4% of the variance. Hazardous exposure (estimate of 0.65)<br />

and physical exerti<strong>on</strong> (0.49) seem to have the lowest impact and <strong>on</strong>ly explain 2% and 5% of the<br />

variance respectively. This might be due to the specific work envir<strong>on</strong>ment of a sales representative,<br />

as it seems they are not necessarily exposed to hazardous circumstances and physical exerti<strong>on</strong>.<br />

This research study therefore indicates that for sales representatives there is a str<strong>on</strong>ger relati<strong>on</strong>ship<br />

between QWL and Managerial Practices than between QWL and their Pers<strong>on</strong>ality c<strong>on</strong>structs.<br />

6. Discussi<strong>on</strong><br />

This research study can be seen as an exploratory attempt to test an integrated model c<strong>on</strong>sisting of<br />

Managerial Practices, Pers<strong>on</strong>ality aspects and QWL. In particular, the aim of this study was to<br />

investigate the implied theoretical relati<strong>on</strong>ship between the dimensi<strong>on</strong>s making up the Organisati<strong>on</strong>al<br />

Trust c<strong>on</strong>struct and those which form the QWL c<strong>on</strong>struct. The results of the analysis indicate a<br />

positive relati<strong>on</strong>ship between QWL and Managerial Practices (0.68) but a lower relati<strong>on</strong>ship with the<br />

Pers<strong>on</strong>ality c<strong>on</strong>structs (0.54).<br />

Martins (2000) and V<strong>on</strong> der Ohe et al (2004) found agreeableness to be a significant manifestati<strong>on</strong> of<br />

the Big Five Pers<strong>on</strong>ality aspects. This was c<strong>on</strong>firmed by the results of this research. Also in<br />

accordance with Martins’s (2000) research, it seems that the Pers<strong>on</strong>ality aspects have a lower impact<br />

<strong>on</strong> Organisati<strong>on</strong>al Trust than Managerial Practices.<br />

7. C<strong>on</strong>clusi<strong>on</strong>s and recommendati<strong>on</strong>s<br />

From the empirical results the assumpti<strong>on</strong> can be made that if an organisati<strong>on</strong> intends to improve the<br />

satisfacti<strong>on</strong> levels of sales representatives, the focus should be <strong>on</strong> improving the Managerial<br />

Practices and QWL dimensi<strong>on</strong>s. In this envir<strong>on</strong>ment a focus <strong>on</strong> the “correct” pers<strong>on</strong>ality types will not<br />

have a great influence <strong>on</strong> Organisati<strong>on</strong>al Trust or positively influence QWL. Research results<br />

regarding the Organisati<strong>on</strong>al Trust c<strong>on</strong>struct have been supported by research carried out by Martins<br />

(2000), Martins and Martins (2002), Martins and V<strong>on</strong> der Ohe (2002), V<strong>on</strong> der Ohe et al (2004) and<br />

V<strong>on</strong> der Ohe and Martins (2010).<br />

References<br />

Bagraim, J. J. and Hime, P. (2007). The dimensi<strong>on</strong>ality of Workplace Interpers<strong>on</strong>al Trust and its relati<strong>on</strong>ship to<br />

Workplace Affective Commitment. SA Journal of Industrial Psychology, Vol.33, No 3, pp 43-48.<br />

Ballou, B. and Godwin, N. H. (2007). Quality of "Work Life": Have you invested in your organisati<strong>on</strong>'s future?<br />

Strategic Finance, October, pp 41-45.<br />

Barrick, M. R. and Mount, M. K. (1991). The Big Five Pers<strong>on</strong>ality dimensi<strong>on</strong>s and job performance: A metaanalysis.<br />

Pers<strong>on</strong>nel Psychology, Vol. 44, No 1, pp 1-26.<br />

Bews, N. and Rossouw, D. (2002). C<strong>on</strong>temporary organisati<strong>on</strong>al change and the importance of trust. SA Journal<br />

of Industrial Psychology, Vol 28, No.4, pp 2-6.<br />

Bollen, K. A. and L<strong>on</strong>g, J. S. (Eds.). (1993). Testing Structural Equati<strong>on</strong> Models. Newbury Park: Sage.<br />

Bozi<strong>on</strong>elos, N. (2004). The relati<strong>on</strong>ship between dispositi<strong>on</strong> and career success: A British Study. Journal of<br />

Occupati<strong>on</strong>al and Organizati<strong>on</strong>al Psychology, Vol. 77,No.3, pp 403-420.<br />

Büssing, A. (2002). Trust and its relati<strong>on</strong>s to commitment and involvement in work and organisati<strong>on</strong>s. SA Journal<br />

of Industrial Psychology, Vol.28, No.4, pp 36-42.<br />

Carmeli, A. (2005). The relati<strong>on</strong>ship between organisati<strong>on</strong>al culture and withdrawal intenti<strong>on</strong>s and behaviour.<br />

Internati<strong>on</strong>al Journal of Manpower,Vol. 26,No 2, pp 177-195.<br />

Church, T. A. (2000). Culture and pers<strong>on</strong>ality:Toward an integrated cultural trait psychology. Journal of<br />

Pers<strong>on</strong>ality, Vol.68, pp 651-704.<br />

Dijkstra, E. J. and Fred, W. V. (2005). Separate and Joint effects of medium type <strong>on</strong> c<strong>on</strong>sumer resp<strong>on</strong>ses: A<br />

comparis<strong>on</strong> of televisi<strong>on</strong>, print and the internet. Journal of Business Research,Vol. 58, No. 3, pp 377-386.<br />

Esterhuizen, W. and Martins, N. (2008). Organisati<strong>on</strong>al justice and employee resp<strong>on</strong>ses to employment equity.<br />

South African Journal of Labour Relati<strong>on</strong>s, Vol.32, No.2, pp 66-85.<br />

Field, A. (2005). Discovering Statistics Using SPSS (2nd ed.), Sage Publicati<strong>on</strong>s, L<strong>on</strong>d<strong>on</strong>.<br />

Gars<strong>on</strong>, G. D. (2004). Structural Equati<strong>on</strong> Modeling. Retrieved September 8, 2010, from<br />

http://www2.uta.edu/sswmindel/S6367/SEM/Principles%20of%20SEM.pdf<br />

Gellatly, I. R. and Irving, P. G. (2001). Pers<strong>on</strong>ality, Aut<strong>on</strong>omy, and C<strong>on</strong>textual Performance of Managers. Human<br />

Performance, Vol.14, No.3, pp 231-245.<br />

192


Nico Martins and Yolandi van der Berg<br />

Gillespie, N. A. and Mann, L. (2004). Transformati<strong>on</strong>al leadership and shared values: the building blocks of trust.<br />

Journal of Managerial Psychology, Vol.19,No.6, pp 588-607.<br />

Goodstein, L. D. and Lany<strong>on</strong>, R. I. (1999). Applicati<strong>on</strong>s of Pers<strong>on</strong>ality Assessment to the Workplace: A Review.<br />

Journal of Business and Psychology,Vol.13, No.3, pp 291-322.<br />

Hannif, Z., Burgess, J. and C<strong>on</strong>nell, J. (2008). Call centres and the quality of work life: Towards a research<br />

agenda. Journal of Industrial Relati<strong>on</strong>s,Vol.50, No.2, pp 271-284.<br />

Hay, A. (2002). Trust and Organisati<strong>on</strong>al change: An experience from manufacturing. SA Journal of Industrial<br />

Psychology, Vol.28,No.4, pp 40-44.<br />

Huang, T., Lawler, J. and Lei, C. (2007). The effects of Quality of Work Life <strong>on</strong> commitment and turnover<br />

intenti<strong>on</strong>. Social Behavior and Pers<strong>on</strong>ality,Vol. 35,No.6, pp 735-750.<br />

Hurtz, G. M. and D<strong>on</strong>ovan, J. J. (2000). Pers<strong>on</strong>ality and Job Performance: The Big Five Revisited. Journal of<br />

Applied Psychology, Vol.85, pp 869-879.<br />

Judge, T. A., Higgins, C. A. and Cable, D. M. (2000). The Employment Interview: A recent research and<br />

recommendati<strong>on</strong>s for Future Research. Human Resource <strong>Management</strong> Review, Vol.10.No.4, pp 383-406.<br />

Kaushik, N. and T<strong>on</strong>k, M. S. (2008). Pers<strong>on</strong>ality and Quality of Work Life. ICFAI Journal of Organizati<strong>on</strong>al<br />

Behavior,Vol. VII,No.3, pp 34-46.<br />

Kirby, E. L. and Harter, L. M. (2001). Discourses of diversity and quality of work life. <strong>Management</strong><br />

Communicati<strong>on</strong> Quarterly, Vol.15,No.1, pp 121-127.<br />

Kline, P. (1999). Handbook of Psychological Testing (2nd ed.), Routledge, New York.<br />

Kotzé, M. (2005). The nature and development of the c<strong>on</strong>struct "quality of work life". Acta <strong>Academic</strong>a,<br />

Vol.37,No.2, pp 96-122.<br />

Lämsä, A. and Pučėtaitė, R. (2006). Development of organisati<strong>on</strong>al trust am<strong>on</strong>g employees from a c<strong>on</strong>textual<br />

perspective. Business Ethics: A <str<strong>on</strong>g>European</str<strong>on</strong>g> Review, Vol.15,No.2, pp 130-141.<br />

L<strong>on</strong>g, C. P., Sitkin, S. B. and Cardinal, L. B. (2003). Managerial Acti<strong>on</strong> to Build C<strong>on</strong>trol, Trust, and Fairness in<br />

Organizati<strong>on</strong>s: The Effect of C<strong>on</strong>flict. Paper presented at the Internati<strong>on</strong>al Associati<strong>on</strong> of C<strong>on</strong>flict<br />

<strong>Management</strong> <str<strong>on</strong>g>C<strong>on</strong>ference</str<strong>on</strong>g>, pp 1-57. Melbourne, Australia.<br />

Martel, J. P. and Dupuis, G. (2006). Quality of Work Life: Theoretical and Methodological Problems, and<br />

Presentati<strong>on</strong> of a New Model and Measuring Instrument. Social Indicators Research, Vol.77,No 2, pp 333-<br />

368.<br />

Martins, N. (2000). Developing a Trust Model for Assisting <strong>Management</strong> During Change. SA Journal of Industrial<br />

Psychology, Vol.26,No.3, pp 27-31.<br />

Martins, N. and V<strong>on</strong> der Ohe, H. (2002). Trust as a factor in determining how to attract, motivate and retain talent.<br />

SA Journal of Industrial Psychology, Vol.28,No.4, pp 49-57.<br />

May, B. E., Lau, R. S. and Johns<strong>on</strong>, S. K. (1999). A l<strong>on</strong>gitudinal study of quality of work life and business<br />

performance. South Dakota Business Review, Vol.58,No.2, pp 3-7.<br />

Mayer, R. C., Davis, J. H. and Schoorman, F. D. (1995). An integrative model of organisati<strong>on</strong>al trust. Academy of<br />

<strong>Management</strong> Review, Vol.20, pp 709-734.<br />

McEvily, B., Weber, R. A., Bicchieri, C. and Ho, V. T. (2006). Can groups be trusted? An experimental study of<br />

trust in collective entities. In R. Bachmann and A. Zaheer (Eds.), Handbook of Trust Research (pp 52-67),<br />

Edward Elgar Publishing Limited, Cheltanham.<br />

Phares, J. E. (1991). Introducti<strong>on</strong> to Pers<strong>on</strong>ality (3rd ed.). New York: Harper Collins.<br />

Pienaar, J., Rothmann, S. R. and Van de Vijver, F. J. (2007). Occupati<strong>on</strong>al Stress, Pers<strong>on</strong>ality Traits, Coping<br />

Strategies, and Suicide Ideati<strong>on</strong> in the South African Police Service. Criminal Justice and Behavior,<br />

Vol.34,No.2, pp 246-258.<br />

Rathi, N. (2010). Relati<strong>on</strong>ship of Quality of Work Life with Employees' Psychological Well-Being. Internati<strong>on</strong>al<br />

Journal of Business Insights and Transformati<strong>on</strong>, Vol.3,No.1, pp 53-60.<br />

Rothmann, S. and Coetzer, E. P. (2003). The Big Five Pers<strong>on</strong>ality Dimensi<strong>on</strong>s and Job Performance. South<br />

African Journal of Industrial Psychology, Vol.29No.1, pp 68-74.<br />

Rousseau, D. M., Sitkin, S. B., Burt, R. S. and Camerer, C. (1998). Not so different after all: A cross discipline<br />

view of trust. Academy of <strong>Management</strong> Review,Vol. 23,No.3, pp 393-404.<br />

Salgado, J. F. (2002). The Big Five Pers<strong>on</strong>ality Dimensi<strong>on</strong>s and Counterproductive Behaviours. Journal of<br />

Selecti<strong>on</strong> and Assessment, Vol.10,No.1/2, pp 117-125.<br />

Schneider, B. J. and Dachler, P. H. (1978). A Note <strong>on</strong> the Stability of the Job Descripti<strong>on</strong> Index. Journal of<br />

Applied Psychology, Vol.63, No.5, pp 650-653.<br />

Schoorman, F. D., Mayer, R. C. and Davis, J. H. (2007). An Integrative model of Organisati<strong>on</strong>al Trust: Past,<br />

Present and Future. Academy of <strong>Management</strong> Review, Vol.32,No.2, pp 344-354.<br />

Schumacker, R. E. and Lomax, R. G. (2004). A Beginner's Cuide to Structural Equati<strong>on</strong> Modeling (2nd ed.),<br />

Lawrence Erlbaum Associates Inc,New Jersey,<br />

Sekwena, E. (2007). Interacti<strong>on</strong> between work and pers<strong>on</strong>al life: Experiences of Police Officers in the North West<br />

Province. Acta Criminologica, Vol.20,No.4, pp 37-54.<br />

Shaw, W. H. (2005). Business ethics, Thoms<strong>on</strong> Wadsworth, Belm<strong>on</strong>t.<br />

Sirgy, M. J., Efraty, D., Siegel, P. and Lee, D. (2001). A New Measurement of Quality of Work Life (QWL) based<br />

<strong>on</strong> Need Satisfacti<strong>on</strong> and Spillover Theories. Social Indicators Research, Vol.55,No.3, pp 241-302.<br />

Srivastava, A. K. (2008). Effect of Perceived Work Envir<strong>on</strong>ment <strong>on</strong> Employees' Job Behaviour and<br />

Organizati<strong>on</strong>al Effectiveness. Journal of the Indian Academy of Applied Psychology, Vol.34,No.1, pp 47-55.<br />

Thoms<strong>on</strong>, L. and de Bruin, K. (2007). Pers<strong>on</strong>ality as predictor of life balance in South African corporate<br />

employees. Journal of C<strong>on</strong>temporary <strong>Management</strong>, Vol.4, pp 68-85.<br />

193


Nico Martins and Yolandi van der Berg<br />

Thoresen, C. J., Kaplan, S. A., Barsky, A. P., Warren, C. R. and De Cherm<strong>on</strong>t, K. (2003). The Affective<br />

Underpinnings of Job Percepti<strong>on</strong>s and Attitudes: A Meta-Analytic Review and Integrati<strong>on</strong>. Psychological<br />

Bulletin, Vol.129,No.6, pp 914-945.<br />

Tschannen-Moran, M. and Hoy, W. K. (2000). A Multidisciplinary Analysis of the Nature, Meaning, and<br />

Measurement of Trust. Review of Educati<strong>on</strong>al Research, Vol.70,No.4, pp 547-593.<br />

Vacola, M., Tsaousis, I. and Nikolaou, I. (2004). The Role of Emoti<strong>on</strong>al Intelligence and Pers<strong>on</strong>ality Variables <strong>on</strong><br />

Attitudes toward Organisati<strong>on</strong>al Change. Journal of Managerial Psychology, Vol.19,No.2, pp 88-110.<br />

Van der Doef, M. and Maes, S. (1999). The Leiden Quality of work Questi<strong>on</strong>naire: Its c<strong>on</strong>structi<strong>on</strong>, Factor<br />

structure, and Psychometric Qualities. Psychological Reports, Vol.85, pp 954-962.<br />

Van der Doef, M. and Maes, S. (2002). Teacher-specific quality of work versus general quality of work<br />

assessment: A comparis<strong>on</strong> of their validity regarding burnout, (psycho)somatic well-being and job<br />

satisfacti<strong>on</strong>. Anxiety, Stress and Coping, Vol.15.No.4, pp 327-344.<br />

V<strong>on</strong> der Ohe, H. and Martins, N. (2010). Exploring Trust Relati<strong>on</strong>ships during times of change. Journal of Human<br />

Resource <strong>Management</strong>, Vol.8,No.1, pp 1-9.<br />

V<strong>on</strong> der Ohe, H., Martins, N. and Roode, M. (2004, Winter). The influence of credibility <strong>on</strong> employer-employee<br />

trust relati<strong>on</strong>s. South African Journal of Labour Relati<strong>on</strong>s, Vol.28,No.2, pp 4-32.<br />

Walt<strong>on</strong>, R. E. (1973). Quality of Work Life: What is it? Sloan <strong>Management</strong> Review, Fall, pp 11-21.<br />

Wayne, J. H., Musisca, N. and Flees<strong>on</strong>, W. (2004). C<strong>on</strong>sidering the role of pers<strong>on</strong>ality in the work-family<br />

experience: Relati<strong>on</strong>ships of the big five to work-family c<strong>on</strong>flict and facilitati<strong>on</strong>. Journal of Vocati<strong>on</strong>al<br />

Behavior, Vol.64, pp 108-130.Wils<strong>on</strong>, M. G., DeJoy, D. M., Vandenberg, R. J., Richards<strong>on</strong>, H. A. and<br />

McGrath, A. L. (2004). Work Characteristics and Employee Health and Well-Being: Test of a Model of<br />

Healthy Work Organizati<strong>on</strong>. Journal of Occupati<strong>on</strong>al and Organizati<strong>on</strong>al Psychology, Vol.77.No.4, pp 565-<br />

588.<br />

Wright, T. A. and B<strong>on</strong>net, D. G. (2007). Job Satisfacti<strong>on</strong> and Psychological Well-Being as N<strong>on</strong>additive Predictors<br />

of Workplace Turnover. Journal of <strong>Management</strong>, Vol.33,No.2, pp 141-160.<br />

Wyatt, T. A. and Wah, C. Y. (2001). Percepti<strong>on</strong>s of QWL: A Study of Singaporean Employees Development .<br />

Research and Practice in Human Resource <strong>Management</strong>, Vol.9,No.2, pp 59-76.<br />

194


Identificati<strong>on</strong> and Governance of Emerging Ethical Issues<br />

in Informati<strong>on</strong> Systems: Empirical and Theoretical<br />

Presuppositi<strong>on</strong>s<br />

Laurence Masclet and and Philippe Gouj<strong>on</strong><br />

LEGIT, Computer Sciences Faculty, University of Namur (FUNDP)<br />

Namur, Belgium<br />

laurence.masclet@fundp.ac.be<br />

pgo@info.fundp.ac.be<br />

Abstract: Our paper addresses the topic of the c<strong>on</strong>ference, by a critical approach of the “Issues in IS design and<br />

development” and more generally, a critical point of view of the way Informati<strong>on</strong> Systems are created and<br />

managed, and especially how ethics is implemented in IS projects. We seek to address the presuppositi<strong>on</strong>s in IS<br />

ethical practice and how they are related to presuppositi<strong>on</strong>s in current theories of governance. IS are a growing<br />

part of the functi<strong>on</strong>ing of industries nowadays and the identificati<strong>on</strong> and governance of the ethical issues that it<br />

raises become a vital matter for society. This paper will describe a study am<strong>on</strong>g Informati<strong>on</strong> Systems<br />

professi<strong>on</strong>als about how they perceive the emerging ethical issues that are present in new IS projects, how they<br />

acknowledge them and the strategies in place to address them. This is the first part of the research, and it is<br />

where most empirical studies stop. The originality of our research is that it articulates to the empirical qualitative<br />

research (made by <strong>on</strong>line questi<strong>on</strong>naires and follow-up Skype interviews) a theoretical critical perspective <strong>on</strong> the<br />

governance theories that determine ethical strategies in IS. We go back and forth between governance theories<br />

and IS practices to find the presuppositi<strong>on</strong>s that are going <strong>on</strong> in both sides. Most governance theories fail to<br />

address the problem of the actual implementati<strong>on</strong> of their theories. Our diagnosis is that current and traditi<strong>on</strong>al<br />

governance theories (derived from Rawls, Habermas, etc.) fall into some presuppositi<strong>on</strong>s that doesn’t allow them<br />

to address implementati<strong>on</strong> properly. For example, they assume that it is sufficient to come to a c<strong>on</strong>sensus<br />

between stakeholders during the c<strong>on</strong>structi<strong>on</strong> of the norm (following strict procedures to ensure fairness), to<br />

reach legitimacy and to reach as a by-product the applicati<strong>on</strong> of the norms. However, the last implicati<strong>on</strong> is not<br />

necessarily true. The reas<strong>on</strong>s why people accept a norm as valid are not necessarily the reas<strong>on</strong>s to accept its<br />

c<strong>on</strong>clusi<strong>on</strong> as a maxim for acti<strong>on</strong>. This presuppositi<strong>on</strong> comes from a more general rati<strong>on</strong>alist background. To<br />

overcome presuppositi<strong>on</strong>s we propose a more comprehensive governance theory, which takes into account the<br />

c<strong>on</strong>text of applicati<strong>on</strong> of the norms within the c<strong>on</strong>structi<strong>on</strong> of the norms (presuppositi<strong>on</strong>s, values, ways of<br />

thinking, etc. found in the interview).<br />

Keywords: ethics, informati<strong>on</strong> systems, emerging technologies, governance, interview, empirical and theoretical<br />

study, links between theory and practice<br />

1. Introducti<strong>on</strong>: Setting the problem (diagnosis <strong>on</strong> the state of IS and ethical<br />

governance)<br />

A certain focus <strong>on</strong> ethics has become increasingly necessary in the field of Informati<strong>on</strong> systems (Felt,<br />

2005). As stated by Pears<strong>on</strong> et al. (Pears<strong>on</strong>, 1996) there are many reas<strong>on</strong>s for IS professi<strong>on</strong>als to try<br />

to progress in the ethical field, and to actually take a reflexive point of view <strong>on</strong> their own work. The<br />

main reas<strong>on</strong> is probably the growing impact of their jobs <strong>on</strong> society. Indeed, IS professi<strong>on</strong>als have a<br />

resp<strong>on</strong>sibility in the selecti<strong>on</strong> of informati<strong>on</strong> that will lead the decisi<strong>on</strong>-makers to make their decisi<strong>on</strong>.<br />

The selecti<strong>on</strong> of informati<strong>on</strong> is <strong>on</strong>e of the most crucial skill needed from IS professi<strong>on</strong>als. With the<br />

Internet, the situati<strong>on</strong> has reversed. It is not any more the lack of informati<strong>on</strong> that will lead to major<br />

problems, or induce the decisi<strong>on</strong> makers to take wr<strong>on</strong>g decisi<strong>on</strong>s, but the excess of informati<strong>on</strong><br />

available (Wurman, 2000), and the variability of the quality of informati<strong>on</strong>. The quality is often even<br />

difficult to assess – even if there are some rules and clues like the existence and reliability of<br />

references, etc. The Informati<strong>on</strong> System Professi<strong>on</strong>al, and as a matter of fact the Informati<strong>on</strong> System<br />

that he/she manages, is the link between the flow of informati<strong>on</strong> and the decisi<strong>on</strong> maker. The<br />

mechanism of selecti<strong>on</strong> of informati<strong>on</strong> is in his hands, and every bias <strong>on</strong> that mechanism will lead to<br />

troubles.<br />

As a basis for decisi<strong>on</strong>, the selecti<strong>on</strong> of informati<strong>on</strong> needs to be ethical. Not <strong>on</strong>ly to make good<br />

decisi<strong>on</strong>s for the sake of society in general, but also to make useful decisi<strong>on</strong>s for the business firm,<br />

that will not leads to a lot of troubles in the process of acceptati<strong>on</strong> by the community of users and the<br />

society in general. Having ethical decisi<strong>on</strong>s at the level of the IS professi<strong>on</strong>al will avoid bad decisi<strong>on</strong>s<br />

for society, and bad decisi<strong>on</strong> for the company, because both are linked together. In the absence of<br />

acceptati<strong>on</strong> from society, which includes the employees of the business firm and the stakeholders, the<br />

195


Laurence Masclet and and Philippe Gouj<strong>on</strong><br />

decisi<strong>on</strong> will lead to negative reacti<strong>on</strong>s, even if it is totally legal, well-thought, and actually useful for<br />

the users.<br />

So, implementing ethics in Informati<strong>on</strong> Systems management and development seems to be a good<br />

and necessary thing for both civil society and industry. As a c<strong>on</strong>sequence, it needs to be encouraged<br />

by policy makers.<br />

But the will for implementing ethics in IS does not seem to be sufficient. Misunderstandings and<br />

ineffectiveness of ethical assessment and ethical awareness seems to be comm<strong>on</strong> place in the<br />

process. As we will see, there are a lot of presuppositi<strong>on</strong>s that restrain and even stop the process of a<br />

good and fully comprehensive ethical governance of IS. Our approach is to determine the<br />

presuppositi<strong>on</strong>s that are present in both ethical theories and IS practices, with an analysis of<br />

interviews with IS professi<strong>on</strong>als from around the globe. An empirical study has been made by De<br />

M<strong>on</strong>fort University, the partner of the University of Namur in the IDEGOV project (IDEntificati<strong>on</strong> and<br />

GOVernance of emerging ethical issues in Informati<strong>on</strong> Systems), via qualitative interviews by<br />

teleph<strong>on</strong>e or Skype with IS professi<strong>on</strong>als representing every c<strong>on</strong>tinents, and every scale of business.<br />

(The c<strong>on</strong>tact with the IT businesses has been made possible via the help of the IMIS (Institute for the<br />

<strong>Management</strong> of Informati<strong>on</strong> Systems)). There has also been <strong>on</strong>line questi<strong>on</strong>naires that has been<br />

send at the occasi<strong>on</strong> of ETHICOMP 2011 “The Social Impact of Social Computing” in Sheffield<br />

Hallam University in the UK.<br />

Our analysis relies <strong>on</strong> a “grid of analysis” designed before the starting of the interview process, in<br />

order to start the analysis of the interview with a fully grounded theoretical background. The grid of<br />

analysis helped us to found the best questi<strong>on</strong>s to ask the interviewees according to parameters that<br />

we found through our analysis of the limits of the theoretical trend of governance. By making the grid<br />

of analysis and describing our parameters and our findings as clearly as possible, we are aim at<br />

letting our own framing apparent, which seems the <strong>on</strong>ly way to deal with the classic epistemological<br />

problem of applying in our research the same scheme we are criticising in others.<br />

2. Presuppositi<strong>on</strong>s in ethical theories<br />

There seems to be a problem <strong>on</strong> the way ethics is implemented into IS. Our hypothesis is that this<br />

problem does not come <strong>on</strong>ly from a lack of c<strong>on</strong>cern in the IS field. This problem raises more<br />

theoretical issues. This is a problem within ethical theories themselves, and <strong>on</strong> particular how they are<br />

taking into account the possibility of their own implementati<strong>on</strong>. Our point of view is to take ethics not<br />

as a soluti<strong>on</strong> given to professi<strong>on</strong>al, who would <strong>on</strong>ly have to be kind enough to implement them in their<br />

work. Ethics is in itself a questi<strong>on</strong>. It is in itself a problem.<br />

A presuppositi<strong>on</strong> has been going <strong>on</strong> for a l<strong>on</strong>g time in the area of ethics and governance, that the <strong>on</strong>ly<br />

way to “do ethics” in highly differential societies, is to rely <strong>on</strong> rati<strong>on</strong>al procedures. Philosophers like<br />

Habermas, Apel or John Rawls took the rati<strong>on</strong>alisati<strong>on</strong> of the world as a way of resoluti<strong>on</strong> of values<br />

c<strong>on</strong>flict in a growingly heterogeneous world. Max Weber, who famously diagnosed the rati<strong>on</strong>alisati<strong>on</strong><br />

of the world and, as a c<strong>on</strong>sequence, the disappearance of ethics, has not seen the possibility of a<br />

radical change in ethics: the possibility for ethics itself to become formal (Ferry, 2002).<br />

This change in ethics has been d<strong>on</strong>e during the XXth centuries. The central questi<strong>on</strong> of ethics<br />

remains what do we ought to do but with the inflecti<strong>on</strong> of the questi<strong>on</strong>: what procedure can we use to<br />

guarantee that what we do is legitimate? This questi<strong>on</strong> leads to the c<strong>on</strong>structi<strong>on</strong> of theories that<br />

answer to the questi<strong>on</strong> of the c<strong>on</strong>structi<strong>on</strong> of a norm that guarantees legitimacy and avoid values<br />

c<strong>on</strong>flict, has been by the means of rati<strong>on</strong>al argumentati<strong>on</strong> and c<strong>on</strong>sensus (Habermas,1981).<br />

The predominance of rati<strong>on</strong>ality and its ability to resolve any questi<strong>on</strong> has already been put in<br />

questi<strong>on</strong> (Sim<strong>on</strong>, 1972), but the trust in rati<strong>on</strong>al procedure, and the use of procedural theories are still<br />

very much alive, especially when talking about the c<strong>on</strong>structi<strong>on</strong> of the norm and its applicati<strong>on</strong> to<br />

technical fields. The questi<strong>on</strong> of the reas<strong>on</strong> why this is the case cannot possibly be discussed here.<br />

What is relevant to us is the impact of that prevalence of procedural thinking in the questi<strong>on</strong> of ethical<br />

implementati<strong>on</strong>.<br />

The problem of the ethical theories implementati<strong>on</strong> is in the theory itself, which lacks at taking into<br />

account that problem in its very c<strong>on</strong>structi<strong>on</strong>. The applicati<strong>on</strong> of the norm is supposed to come<br />

naturally from the legitimacy of the norm.<br />

196


Laurence Masclet and and Philippe Gouj<strong>on</strong><br />

“Procedural ethical theories, in particular, first set themselves the task of indicating a<br />

procedure through which norms and modes of acti<strong>on</strong> can be rati<strong>on</strong>ally grounded or<br />

criticized, as the case may be. Because they must deal with this task separately, the<br />

impartial applicati<strong>on</strong> of valid principles and rules arises <strong>on</strong>ly as a subsequent problem.”<br />

(Habermas, 1991)<br />

The important part of the problem for philosopher like Habermas and Rawls, who are the main<br />

influence in c<strong>on</strong>temporaneous political theories, is not the applicati<strong>on</strong> of the norm. That applicati<strong>on</strong> is<br />

a subsequent problem, a problem that can be fixed afterwards.<br />

The separati<strong>on</strong> of the applicati<strong>on</strong> level from the norms c<strong>on</strong>structi<strong>on</strong> level seems to be <strong>on</strong>e of the<br />

sources of the lack of communicati<strong>on</strong> between ethics and technology. This separati<strong>on</strong> is due to a<br />

number of presuppositi<strong>on</strong>s (Lenoble and Maesschalck, 2003). The first presuppositi<strong>on</strong> is to assume<br />

the intenti<strong>on</strong> to adopt a norm is sufficient to reach the effectiveness of the norm. This presuppositi<strong>on</strong><br />

is called “intenti<strong>on</strong>alist” by Maesschalck and Lenoble. However, it is well known that the rati<strong>on</strong>al will is<br />

very often overcome by other instincts, other preoccupati<strong>on</strong>s. The Greeks called it Akrasia, the<br />

Christians wrote entire treaty about the weakness of the will (Saarinen 1994). Why do c<strong>on</strong>temporary<br />

polititical theories assume that <strong>on</strong>ce norms are rati<strong>on</strong>ally c<strong>on</strong>structed, and accepted by rati<strong>on</strong>al<br />

c<strong>on</strong>sensus, people will act according to them?<br />

Another presuppositi<strong>on</strong> that we can find in most governance theories is the mentalist presuppositi<strong>on</strong>,<br />

which is the presuppositi<strong>on</strong> that the c<strong>on</strong>diti<strong>on</strong>s that determine the effectiveness of norms are linked to<br />

rules presupposed within the mind and c<strong>on</strong>sequently are supposed to be a functi<strong>on</strong> of mental<br />

capacities. Since the mental capacities are independent of the external c<strong>on</strong>text of the subject, most<br />

procedural governance theories ignore the questi<strong>on</strong> of the effectiveness of the norm. In other words,<br />

they think that the effectiveness of the implementati<strong>on</strong> of norms is not a questi<strong>on</strong>, because it is not<br />

dependant of external governance, but is the intern result of the norm itself. The mentalist<br />

presuppositi<strong>on</strong> is then thinking that the existence of norms is enough to activate mechanism in the<br />

mind that will assure the effectiveness of implementati<strong>on</strong>.<br />

The third presuppositi<strong>on</strong> isolated by Maesschalck et al. is the schematising presuppositi<strong>on</strong>. It relies <strong>on</strong><br />

the mind having a set of rules (or schemes, in Kant’s words), that predetermines the effect of a norm,<br />

and does not depend <strong>on</strong> any exterior c<strong>on</strong>text (to that of the thinker). This is comm<strong>on</strong>ly seen when<br />

participants in a participatory approach come to the setting with their own particular ethical framing, or<br />

with some prec<strong>on</strong>cepti<strong>on</strong>s as to what ethical issues might arise. (Rainey and Gouj<strong>on</strong>, 2009)<br />

Another presuppositi<strong>on</strong>, identified by another author, Jean-Marc Ferry, is that most ethical theories<br />

and governance arrangements ignore that the c<strong>on</strong>diti<strong>on</strong> why we accept a norm is not equal to the<br />

c<strong>on</strong>diti<strong>on</strong> of its justificati<strong>on</strong>. (Ferry, 2002), which means that we can accept a norm for other reas<strong>on</strong>s<br />

than its rati<strong>on</strong>al justificati<strong>on</strong>, and that we can accept a norm without agreeing with its justificati<strong>on</strong>.<br />

Those presuppositi<strong>on</strong>s can be found in traditi<strong>on</strong>al ethical governance theories. However, the more<br />

c<strong>on</strong>temporary <strong>on</strong>e have embraced those presuppositi<strong>on</strong>s as well, even if they do shape it in another<br />

way. We can say that the disregard in ethical theories for the problem of the resoluti<strong>on</strong> of ethical<br />

issues is related to those presuppositi<strong>on</strong>s. There is still, and maybe more than ever, a separati<strong>on</strong><br />

between the problem of norms c<strong>on</strong>structi<strong>on</strong> and the problem of norms applicati<strong>on</strong>, which leads to a<br />

blind point c<strong>on</strong>cerning the c<strong>on</strong>text of applicati<strong>on</strong> of the norms in governance theories (including the<br />

<strong>on</strong>e from Maeschalck et al.). The third presuppositi<strong>on</strong> we can find is the presuppositi<strong>on</strong> that the<br />

determinati<strong>on</strong> of ethical issues and the c<strong>on</strong>structi<strong>on</strong> of a norm is sufficient to resolve every ethical<br />

issue. This last presuppositi<strong>on</strong> is comm<strong>on</strong> place in the ethical governance project about technology.<br />

Indeed, a lot of the project are focus <strong>on</strong>ly <strong>on</strong> identificati<strong>on</strong> of ethical issues in a particular technology<br />

or technical field, and the project stops there, assuming that <strong>on</strong>ce the ethical issues has been<br />

identified, it will be naturally taken in charge and resolve. This is obviously related to the intenti<strong>on</strong>alist<br />

presuppositi<strong>on</strong>. Our research has shown that the awareness of an issue is not indeed sufficient to<br />

resolve it (Masclet and Gouj<strong>on</strong>, 2011). Identificati<strong>on</strong> can be a first step, but it is often misleading,<br />

because it can be use as an alibi for industries and policy makers.<br />

Knowing where the problem is, where there is a space for a potential ethical problem does not in itself<br />

resolve the problem. It may raise awareness and carefulness, but that is a c<strong>on</strong>sequence that cannot<br />

be taken for granted, and above all, that is not necessarily sufficient to avoid the issues. Some issues<br />

need more to deal with it than awareness. Awareness is a good first step, but it cannot be assume<br />

197


Laurence Masclet and and Philippe Gouj<strong>on</strong><br />

that, because people are aware of an issue, they necessarily will take care of it, and make sure to<br />

avoid to actualise the potential issue. This is all c<strong>on</strong>nected with the theory of bounded rati<strong>on</strong>ality.<br />

We cannot expect people to act fully rati<strong>on</strong>ally. It is not because <strong>on</strong>e is aware of an issue that he/she<br />

will act to resolve it. There are too many interests, too many c<strong>on</strong>textual incentives <strong>on</strong> the policy<br />

makers, <strong>on</strong> the technological developers and <strong>on</strong> every stakeholder, for us to assume that it is enough<br />

to point out an issue to reach a soluti<strong>on</strong>. As we will see, this presuppositi<strong>on</strong> is shared by a lot of IS<br />

professi<strong>on</strong>al, as well as with ethics professi<strong>on</strong>als. This, in a way, makes the problem of the<br />

ineffectiveness of the norms worst, because, if issue identificati<strong>on</strong> is indeed the <strong>on</strong>ly area that is<br />

shared by IS professi<strong>on</strong>als and philosophers, it can be seen as a nice and easy way of ethics<br />

implementati<strong>on</strong> by everybody in good will. However the identificati<strong>on</strong> of ethical issues does not<br />

resolve them (Van den Hoven, 2008), and can even hide a more general problem in the field and lead<br />

to c<strong>on</strong>textual blindness. Indeed, making a list of issues that professi<strong>on</strong>al should beware make them<br />

less aware of other c<strong>on</strong>textually induced issues. It takes the resp<strong>on</strong>sibility off of the professi<strong>on</strong>al<br />

shoulders, and makes them unreactive to possible ethical disaster that would not be in the ethical<br />

issue list because it could not be predicted. This is the same critique as the critique we can do against<br />

codes of c<strong>on</strong>duct and ethical guidelines.<br />

Before discussing the best ways to overcome those presuppositi<strong>on</strong>s and the negative impact they<br />

have <strong>on</strong> the relati<strong>on</strong>ship between ethics and technology, we would like to pass to the other side of the<br />

gap, and look at the presuppositi<strong>on</strong>s about ethics in the IS field. This secti<strong>on</strong> will rely <strong>on</strong> the interview<br />

d<strong>on</strong>e for the IDEGOV project by the DMU University (Wakunuma and Stahl, 2011), and the analysis<br />

of the interviews that we have d<strong>on</strong>e in collaborati<strong>on</strong> (Masclet and Gouj<strong>on</strong>, 2012).<br />

3. Presuppositi<strong>on</strong>s in IS<br />

We said in our introducti<strong>on</strong> that ethics was (or at least should be) profitable for enterprises and that<br />

IS, as any technology, would be better implemented and safest, not <strong>on</strong>ly <strong>on</strong> human side, but also <strong>on</strong><br />

the ec<strong>on</strong>omic point of view, if it implement ethical behaviour. Now, we have to see what is really been<br />

d<strong>on</strong>e in IS c<strong>on</strong>cerning ethics.<br />

As shown in the interviews we did for IDEGOV (Wakunuma and Stahl, 2012), informati<strong>on</strong> Systems<br />

professi<strong>on</strong>als are aware to a certain extend to ethics. They usually implement a top-down approach<br />

<strong>on</strong> ethical issue. The main soluti<strong>on</strong> given to ethical issues seems to be awareness, as we menti<strong>on</strong>ed<br />

earlier. It is assumed that informati<strong>on</strong> is the best way to deal with ethical issues. For example, to deal<br />

with the ethical issues that can provoke the installati<strong>on</strong> of camera in shops or banks, it seems<br />

sufficient to explain to the users why the cameras are installed.<br />

This is a remarkable example of how the principles of bioethics have migrated into every<br />

technological field. This is due to the development of the bioethical field. It seems that informati<strong>on</strong> and<br />

free c<strong>on</strong>sent are seen now for what ethics is about (even if, in our example, the c<strong>on</strong>sent is not even<br />

invoked). This lead to top-down management of ethical questi<strong>on</strong>s, which are treated by the chief or at<br />

least the pers<strong>on</strong> “resp<strong>on</strong>sible” (pers<strong>on</strong>al director, or IT manager). Society or users are not involved<br />

per se in the mechanism of treating with an ethical issue. Reducing the ethical strategies to<br />

informati<strong>on</strong> and c<strong>on</strong>sent is also a way for the IS professi<strong>on</strong>als not to change their systems or their<br />

technologies, so, not to change their point of view. The fault is put <strong>on</strong> the ignorance of the user and<br />

the society. This prevalence of awareness and informati<strong>on</strong> is also due to the very positive bias that<br />

bring the noti<strong>on</strong> of informati<strong>on</strong> nowadays (noti<strong>on</strong> that come from the development of cybernetics and<br />

mathematical theories of informati<strong>on</strong>, to invade every sphere of society within years, to become part<br />

of the new ideology of the modern world (Wiener, 1950)). Even if we found reflexivity in the ethical<br />

behaviour of the IS professi<strong>on</strong>als, it is always restricted to first-level reflexivity (Argyris and Schön,<br />

1978), which is thinking about the reas<strong>on</strong> of its own acti<strong>on</strong>, but never a sec<strong>on</strong>d level-reflexivity, which<br />

would involve a critical thinking about the presuppositi<strong>on</strong>s and the c<strong>on</strong>structi<strong>on</strong> behind <strong>on</strong>e’s own<br />

acti<strong>on</strong>s.<br />

Moreover, the governance strategies in place seem to never use co-c<strong>on</strong>structi<strong>on</strong> or even c<strong>on</strong>sultati<strong>on</strong>.<br />

It usually use a standard model (Jolly, 2001), which is a expert-driven model, who provide the<br />

normativity from a (supposed) objective point of view, or the revised standard model, which is the<br />

extensi<strong>on</strong> of the standard model, with a c<strong>on</strong>siderati<strong>on</strong> of the social c<strong>on</strong>structi<strong>on</strong> of the problem, but in<br />

a very external point of view, as in risk management, for example. In this revised model, public<br />

influence and participati<strong>on</strong> in risk management are c<strong>on</strong>sidered with great suspici<strong>on</strong>. In other words,<br />

198


Laurence Masclet and and Philippe Gouj<strong>on</strong><br />

the c<strong>on</strong>text is c<strong>on</strong>sidered, but reduced to risk assessment according to the main framing. We do not<br />

say that ethical problems are not discussed in IS field and IS companies or companies that use IS.<br />

But, when companies want to implement ethics in their practices, they very often use ethical tools that<br />

are much dec<strong>on</strong>textualised, and that does not take into account the social c<strong>on</strong>structi<strong>on</strong> of the ethical<br />

problem, or the questi<strong>on</strong> of the acceptance of their product by the users in the c<strong>on</strong>structi<strong>on</strong> of their<br />

norms of behaviour. IS professi<strong>on</strong>al very often relies <strong>on</strong> their own experience. And when they do have<br />

discussi<strong>on</strong> about ethics, it is very often within other member of the team. They do not seem keen <strong>on</strong><br />

using the experience of people from civil society, or even from other professi<strong>on</strong> related to theirs.<br />

There is however a noticeable excepti<strong>on</strong> with law-people, who are often c<strong>on</strong>sulted. Nevertheless, law<br />

is distinct from ethics because there always will be a difference between what we ought to do, and<br />

what is allowed. The reducti<strong>on</strong> of ethics to law is a quite popular presuppositi<strong>on</strong>. It leads to<br />

misunderstandings about ethics and can have bad c<strong>on</strong>sequences. The reducti<strong>on</strong> may have been<br />

induced by the proceduralisati<strong>on</strong> of ethics, which became more and more a matter of compliance.<br />

However, the questi<strong>on</strong> is not the same. Lawyers can tell what is allow or not in a certain c<strong>on</strong>text.<br />

Being ethical involves something else. It is not <strong>on</strong>ly a matter of compliance, it cannot be externally<br />

imposed. That particularity is what makes it so difficult to implement: the very term of “implementati<strong>on</strong>”<br />

is misused. Ethics cannot be implemented. It has to come from the pers<strong>on</strong>s. (Masclet and Gouj<strong>on</strong>,<br />

2012)<br />

But that is not to say that ethical theories have no impact. Even if ethics requires somehow a will, that<br />

will is a necessary c<strong>on</strong>diti<strong>on</strong>, but not a sufficient c<strong>on</strong>diti<strong>on</strong>. That is where ethical governance theories<br />

can help.<br />

4. Hint for a comprehensive soluti<strong>on</strong><br />

How can ethics help? In our research, we found various closures and presuppositi<strong>on</strong>s from ethical<br />

people and technology people. There is also a gap between the two communities, due to various<br />

reas<strong>on</strong>s (difference of “jarg<strong>on</strong>”, disinterest for implementati<strong>on</strong> from the philosophers, disinterest for<br />

ethics and assimilati<strong>on</strong> to laws from IS professi<strong>on</strong>als, c<strong>on</strong>text of separati<strong>on</strong> between “human”<br />

sciences and “pure” sciences, notably in educati<strong>on</strong>, and so <strong>on</strong>). The role of ethics nowadays seems to<br />

be opening framings and allowing full reflexivity (Schön, 1983) for everybody, in order to make b<strong>on</strong>ds<br />

between the two disciplines. This approach involves reflexivity <strong>on</strong> the trends of ethical and<br />

governance researches itself. This first task is a task that has always been prevalent in the work of<br />

philosophers. Reflecti<strong>on</strong> <strong>on</strong> its own activity is at the heart of philosophy. Putting that task back in the<br />

centre of the research in ethics is a good first step. However, the presuppositi<strong>on</strong>s in ethics are not<br />

really about reflexivity per se. The problem might be that ethical researchers are too focused <strong>on</strong> the<br />

theories they are elaborating. A good balance between reflexivity in ethics and a renewed interest <strong>on</strong><br />

the c<strong>on</strong>text of applicati<strong>on</strong> of the theories has to be found. Our diagnosis, after our research for EGAIS<br />

and IDEGOV, is that ethical theories have to take into account the c<strong>on</strong>text of applicati<strong>on</strong> of their own<br />

theories, within the theory itself, which is to say, include an opening, in the theories, to challenges<br />

from the field to which they want to apply their theories, and to society in general. The validity of the<br />

theories, insure by ethical procedures, does not necessarily mean that the theory will actually be<br />

applicable. There is more to take into account than the legitimacy of the procedure to create norms. In<br />

the procedure, it should be acknowledge that the validity of a norm is not always sufficient to insure<br />

it’s acceptati<strong>on</strong>, and furthermore, that somebody can accept rati<strong>on</strong>ally a norms, and even help to<br />

create it within a discussi<strong>on</strong> framed by ethical procedure (taking into account every argument, law of<br />

the best argument, and so <strong>on</strong>), will not necessary take that norm as a maxim for acti<strong>on</strong> (Ferry, 2002).<br />

There is a lot of reas<strong>on</strong> for that gap in ethical theories. One reas<strong>on</strong> is the rejecti<strong>on</strong> of value from the<br />

field of norm elaborati<strong>on</strong>. Ejecting values from the discussi<strong>on</strong> does however not seems to be sufficient<br />

to avoid them in the field of applicati<strong>on</strong> of the norms. On the c<strong>on</strong>trary, values, beliefs, life experiences,<br />

individual point of view, and everything that influence the behaviour without complying with the<br />

exigencies of the rati<strong>on</strong>al discussi<strong>on</strong>, can come disrupt the norm applicati<strong>on</strong>. Opening the theories of<br />

norms c<strong>on</strong>structi<strong>on</strong> to c<strong>on</strong>text is <strong>on</strong>e soluti<strong>on</strong> that can be handled by ethics. What does that mean<br />

exactly? First, that implies to overcome the gap between ethical theories and the so-called field<br />

implementati<strong>on</strong>. As we said, this cannot be about implementati<strong>on</strong> of ethics into a field –any field. The<br />

process of reaching ethical behaviour and ethical innovati<strong>on</strong> and technological development has to<br />

embrace ethics as a collective task, where everybody is resp<strong>on</strong>sible and c<strong>on</strong>tributor of every step of<br />

the process, from the first idea to the launching of a new technology in the society and the analysis of<br />

its impacts, its acceptati<strong>on</strong> and its use.<br />

199


Laurence Masclet and and Philippe Gouj<strong>on</strong><br />

There is a political call, notably from the <str<strong>on</strong>g>European</str<strong>on</strong>g> commissi<strong>on</strong>, to improve the mechanism of<br />

c<strong>on</strong>sultati<strong>on</strong> and participati<strong>on</strong> at the development stage of a new technology. However, we believe<br />

that, even if it is of course necessary, the framework in which they want to improve the procedure<br />

does not achieve a full reflexivity from every stakeholder, and, as a c<strong>on</strong>sequence, stay very external.<br />

The framework used keeps the process of norm c<strong>on</strong>structi<strong>on</strong> outside the technology project, which<br />

has to applied the rules at <strong>on</strong>e point, show that the ethical procedures has been respected before the<br />

ethical review process, as see in figure 1 (to some extend, and taking into account the necessary<br />

simplificati<strong>on</strong> of a schema). There is, in this process, a mechanism of participati<strong>on</strong> from the<br />

technological community in the process of the c<strong>on</strong>structi<strong>on</strong> of the norm, but this participati<strong>on</strong> is much<br />

dec<strong>on</strong>textualised. It does not involve a particular project, and does not, as a c<strong>on</strong>sequence rely <strong>on</strong> the<br />

particular knowledge of the scientific that would participate, who is a stakeholder as anybody else (to<br />

the extend that he/she may raise better arguments, that can be taken into account). The problem is<br />

that, even if the ethical process involves scientists and members of technology project as stakeholder<br />

and partner of the discussi<strong>on</strong> that will lead to the norms, the norms will still be applied in a very<br />

external way, at a very specific point of the development of the technology (usually at the end). This is<br />

related to the limits of the use of experts and their cognitive closure (Masclet and Gouj<strong>on</strong>, 2011),<br />

Relati<strong>on</strong>s between ethics and technology projects<br />

(how it works now)<br />

Technology<br />

project<br />

Technology<br />

First idea<br />

community<br />

Ethical<br />

review<br />

Participati<strong>on</strong><br />

(as stakeholder)<br />

Ethics<br />

community<br />

Process of norms<br />

c<strong>on</strong>structi<strong>on</strong><br />

Figure 1: Relati<strong>on</strong> between ethics and technology projects, as it is now<br />

What we are trying to design, in our project, is a way of implying ethical thinking at every steps of the<br />

development of a project. For that, we cannot <strong>on</strong>ly rely <strong>on</strong> ethical expert. We have to involve the<br />

scientist and the developers in the process from the beginning (figure 2).<br />

Relati<strong>on</strong> between ethics and a technology project (to be<br />

achieved)<br />

Process of norm<br />

c<strong>on</strong>structi<strong>on</strong>, integrating<br />

the c<strong>on</strong>text of the<br />

particular project and its<br />

c<strong>on</strong>text of applicati<strong>on</strong><br />

A technology<br />

project.<br />

Ethical and<br />

technological<br />

communities,<br />

working in c<strong>on</strong>stant<br />

collaborati<strong>on</strong><br />

Figure 2: Relati<strong>on</strong> between ethics and a technology project, how it should work<br />

200


Laurence Masclet and and Philippe Gouj<strong>on</strong><br />

It has to involve training and learning mechanisms, it is also necessary to rec<strong>on</strong>nect the communities<br />

of philosophy and science, and overcome the presuppositi<strong>on</strong>s that are going <strong>on</strong> in both side.<br />

5. C<strong>on</strong>clusi<strong>on</strong><br />

We have to find ways to open the framing of the developers and the users of technologies, for them to<br />

act ethically at every step of their project, but we also have to find a way to open the framing of the<br />

philosophers of governance.<br />

The way of doing ethics has an impact <strong>on</strong> the ethical theories. Of course, we do not say that ethics<br />

has to come <strong>on</strong>ly straight from the scientists and project developers and that ethical theories do not<br />

matter. The input of ethics in technological development would be to create theories that could backup<br />

ethical discussi<strong>on</strong> and give process to reach fairness and reflexivity, and <strong>on</strong> the other hand, the<br />

input of professi<strong>on</strong>als could be to teach ethical theories <strong>on</strong> how to take into account the c<strong>on</strong>text of<br />

applicati<strong>on</strong> of the theories. This implies not <strong>on</strong>ly the specificities of a particular field or project, but also<br />

the particularity of the people who c<strong>on</strong>duct those projects and the people who will use the technology.<br />

A way of doing that is to open ethics and governance theories to other level of discourse than pure<br />

argumentati<strong>on</strong> and reas<strong>on</strong>. We have to think about a way to include narrati<strong>on</strong>, to include<br />

interpretati<strong>on</strong>s and rec<strong>on</strong>structi<strong>on</strong>, to include values and c<strong>on</strong>text into the procedure of governance.<br />

This is currently being tested as a theory called “comprehensive proceduralism”, which takes some of<br />

the hints toward soluti<strong>on</strong> we have developed in this article into a more elaborate theory.<br />

Our approach involve a c<strong>on</strong>stant reflexivity from every actor involve, in order to understand better the<br />

framing for acti<strong>on</strong>, the c<strong>on</strong>textual incentives, and the background theories that inform practices.<br />

Opening framings can also be d<strong>on</strong>e by involving other actors. This is why we are currently analysing<br />

the impact of that civil society organisati<strong>on</strong> (CSO) can make <strong>on</strong> technological project, by using our<br />

method of exploring the background prepositi<strong>on</strong> that inform noti<strong>on</strong> like “participati<strong>on</strong>”.<br />

Acknowledgments<br />

This article relies partially <strong>on</strong> studies d<strong>on</strong>e for the IDEGOV project by the University of Namur.<br />

IDEGOV states for “Identificati<strong>on</strong> and Governance of emerging ethical issues in Informati<strong>on</strong> systems”.<br />

The project has been founded by the CIGREF foundati<strong>on</strong> between 2011 and 2012 and involves the<br />

Laboratory for Ethical Governance of Informati<strong>on</strong> Technology (LEGIT) in the University of Namur,<br />

Belgium, and the Center for Computing and Social Resp<strong>on</strong>sibility in De M<strong>on</strong>tfort University in<br />

Leicester, UK.<br />

References<br />

Argyris, C and Schön, D. A. (1978), Organisati<strong>on</strong>al Learning, Vol.1 A Theory of Acti<strong>on</strong> Perspective, Reading, MA,<br />

Addis<strong>on</strong>, Wesley<br />

Brey, P. (1999), “Method in Computer Ethics: Towards a Multi-Level Interdisciplinary Approach” in Ethics and<br />

Informati<strong>on</strong> Technology, 2:3, 1-5.<br />

Felt Ulrike, (2005), Science, Technology and Democracy Report to the <str<strong>on</strong>g>European</str<strong>on</strong>g> Commissi<strong>on</strong>, Science in<br />

society-Forum, Brussels March 9-11, 2005 Sessi<strong>on</strong> 2<br />

Ferry, Jean-Marc, Valeurs et Normes, (2002), La questi<strong>on</strong> de l’éthique, Bruxelles, Editi<strong>on</strong> de l’université de<br />

Bruxelles<br />

Habermas, Jürgen, (1981), The Theory of Communicative Acti<strong>on</strong><br />

Habermas, Jürgen, (1991), Erläuterungen zur Diskursethik, Frankfurt am Main, Suhrkamp<br />

H<strong>on</strong>neth Axel (1996), Struggle for Recogniti<strong>on</strong>. The Moral Grammar of Social C<strong>on</strong>flicts, Polity Press<br />

Joly, P.-B. (2001) Les OGM entre la science et le public? Quatre modèles pour la gouvernance de l'innovati<strong>on</strong> et<br />

des risques, Ec<strong>on</strong>omie Rurale, 266.<br />

Laud<strong>on</strong>, K., Laud<strong>on</strong>, J., <strong>Management</strong> des systèmes d’informati<strong>on</strong> (11 e éditi<strong>on</strong>) Pears<strong>on</strong> Educati<strong>on</strong>, 2011.<br />

Lenoble, Jacques and Maesschalck, Marc, (2003), Toward a Theory of Governance, the Acti<strong>on</strong> of Norms (trad. J.<br />

Paters<strong>on</strong>), Kluwer Law Internati<strong>on</strong>al<br />

Masclet Laurence, Gouj<strong>on</strong>, Philippe, (2011), IDEGOV D.1.1. Grid of Analysis, CIGREF Foundati<strong>on</strong>.<br />

Masclet Laurence, Gouj<strong>on</strong>, Philippe, (2011), IDEGOV D.3.2. Model of current and emerging governance<br />

strategies, Map of governance and ethics, CIGREF Foundati<strong>on</strong>.<br />

Saarinen, Risto, (1994) Weakness of the will in medieval though: from Augustine to Buridan, EJ. Brill, the<br />

Netherlands.<br />

Schön, D., The Reflective Practiti<strong>on</strong>er. How professi<strong>on</strong>als think in acti<strong>on</strong>, L<strong>on</strong>d<strong>on</strong>: Temple Smith, 1983.<br />

Sim<strong>on</strong> Herbert A, (1972) “theories of bounded rati<strong>on</strong>ality”, in CB. McGuire and Ray Radned (Eds) Decisi<strong>on</strong> and<br />

Organizati<strong>on</strong>, North-Holland Publishing Company<br />

201


Laurence Masclet and and Philippe Gouj<strong>on</strong><br />

Pears<strong>on</strong>, J.M., Crosby, L., Shim, J.P., (1996) Modeling the relative importance of ethical behavior criteria: A<br />

simulati<strong>on</strong> of informati<strong>on</strong> systems professi<strong>on</strong>als’ ethical decisi<strong>on</strong>s. J. Startegic Inform. Systems 5 (4), 275-<br />

291<br />

Rainey Stephen, Gouj<strong>on</strong> Philippe, (2009) EGAIS 4.1 Existing Soluti<strong>on</strong>s to the Ethical Governance Problem and<br />

Characterisati<strong>on</strong> of their Limitati<strong>on</strong>s<br />

Rawls, J., A Theory of Justice, Harvard University Press, Cambridge (Mass.), 1971<br />

Van den Hoven, Jeroen, (2008) “Moral Methodology and Informati<strong>on</strong> Technology,” in The Handbook of<br />

Informati<strong>on</strong> and Computer Ethics, K. Himma and H. Tavani (eds.), Hoboken, NJ: Wiley, pp. 49-68.<br />

Wakunuma, Kutoma, Stahl, Bernd, (2011), IDEGOV D.1.2. Data Collecti<strong>on</strong> Strategy Document, CIGREF<br />

Foundati<strong>on</strong>.<br />

Wurman, Richard, (2000) Informati<strong>on</strong> anxiety 2, Pears<strong>on</strong> Educati<strong>on</strong>.<br />

Wiener, Norbert (1950), The Human Use of Human Beings: cybernetics and society, The Riverside Press<br />

(Hought<strong>on</strong> Mifflin Co.)<br />

202


Breaking C<strong>on</strong>sensus in IS Evaluati<strong>on</strong>s: The Agitati<strong>on</strong><br />

Workshop<br />

John McAvoy, Tadhg Nagle and David Samm<strong>on</strong><br />

Business Informati<strong>on</strong> Systems, University College Cork, Ireland<br />

j.mcavoy@ucc.ie<br />

t.nagle@ucc.ie<br />

dsamm<strong>on</strong>@afis.ucc.ie<br />

Abstract: As researchers evaluate organisati<strong>on</strong>s, there is a desire for a c<strong>on</strong>sensus from those within the<br />

organisati<strong>on</strong>s who are participating in the research. A comm<strong>on</strong> c<strong>on</strong>sensual perspective from a team appears to<br />

reflect an optimal state where those being studied have a comm<strong>on</strong> understanding of the current state of events<br />

within the c<strong>on</strong>text of their envir<strong>on</strong>ment. The questi<strong>on</strong> arises, though, whether an evaluati<strong>on</strong> finding c<strong>on</strong>sensus<br />

reflects the reality: there are a variety of reas<strong>on</strong>s why a comm<strong>on</strong> understanding may be false c<strong>on</strong>sensus. This<br />

paper proposes an evaluati<strong>on</strong> method where, when symptoms of problems such as groupthink are identified, a<br />

c<strong>on</strong>sensus of perspectives is challenged before they are c<strong>on</strong>sidered valid. This is achieved in a workshop where<br />

participants reflect <strong>on</strong> their own percepti<strong>on</strong> of reality and represent this reality in a matrix of influencing and<br />

relevant factors. The individual matrices are then combined and used to highlight disparities in the participants’<br />

perspectives through a single matrix visualisati<strong>on</strong>. Discussi<strong>on</strong> in the workshop then focusses <strong>on</strong> the areas,<br />

highlighted by the matrix, where differences of perspectives are identified. In effect, the comm<strong>on</strong> understanding<br />

presented by those being evaluated will be challenged, and a new comm<strong>on</strong> understanding will have to be<br />

created.<br />

Keywords: comm<strong>on</strong> understanding, c<strong>on</strong>sensus, workshops, groupthink, evaluati<strong>on</strong><br />

1. Introducti<strong>on</strong><br />

Organisati<strong>on</strong>s are faced with increasing demands to deliver, and evaluati<strong>on</strong>s are used to determine<br />

opportunities for improvement. When evaluating organisati<strong>on</strong>s, there is a desire for c<strong>on</strong>sensus from<br />

the participants as it is assumed that c<strong>on</strong>sensus represents the reality of the organisati<strong>on</strong> because all<br />

participants agree <strong>on</strong> this reality. This aligns somewhat with Richards<strong>on</strong>’s (2003, p.1625) descripti<strong>on</strong><br />

of psychological c<strong>on</strong>structivism where “if the individuals within a group come to an agreement about<br />

the nature and warrant of a descripti<strong>on</strong> of a phenomen<strong>on</strong> or its relati<strong>on</strong>ship to others, these meanings<br />

become formal knowledge.” Others have noted the importance of c<strong>on</strong>sensus both in research and in<br />

practice: Bjorn and Mort<strong>on</strong> (2005) describe how individual perspectives can have a negative impact<br />

which makes agreement difficult, while McMah<strong>on</strong> (2003) notes the importance of c<strong>on</strong>sensus in Agile<br />

software development teams. So while it is acknowledged that c<strong>on</strong>sensus within a team can give a<br />

good representati<strong>on</strong> of reality, there are times when this may not be so,<br />

Questi<strong>on</strong>s must be raised as to whether c<strong>on</strong>sensus is necessarily the optimal goal that evaluators of<br />

organisati<strong>on</strong>s should strive for. For example, Pfeffer and Sutt<strong>on</strong> (2000) illustrate the existence of a<br />

‘knowing-doing gap’, where decisi<strong>on</strong>s are not made rati<strong>on</strong>ally <strong>on</strong> the basis of related known facts, but<br />

are shaped by normative or political-cognitive influences. Additi<strong>on</strong>ally, discrepancies between<br />

knowledge as ‘justified true belief’ and acti<strong>on</strong>s taken by a social actor is captured by Argyris and<br />

Sch<strong>on</strong> (1978) in their c<strong>on</strong>ceptualisati<strong>on</strong> of ‘espoused theories’ versus ‘theories in use’. ‘Espoused<br />

theories’ are the beliefs individual’s profess as guiding their behaviours and decisi<strong>on</strong> making, while<br />

‘theories in use’ guide actual behaviour. When evaluati<strong>on</strong>s are c<strong>on</strong>ducted in groups, further problems<br />

arise such as where “various participants attempt[ed] to justify their own positi<strong>on</strong> or to persuade<br />

others to that opini<strong>on</strong>” (Love, 2000, p.431).<br />

One of the major causes of false c<strong>on</strong>sensus within organisati<strong>on</strong>s is groupthink and has been shown to<br />

impact <strong>on</strong> organisati<strong>on</strong>s of differing sizes and goals (cf. Esser, 1998; Leana, 1985; Turner and<br />

Pratkanis, 1998): from small Informati<strong>on</strong> System development teams (cf. McAvoy and Butler, 2009) to<br />

the United States government (cf. Janis, 1972). Groupthink is defined by its originator (Janis, 1972,<br />

p.9) as “a deteriorati<strong>on</strong> of mental efficiency, reality testing, and moral judgement that results from ingroup<br />

pressures.” This is further refined as “the psychological drive for c<strong>on</strong>sensus at any cost”<br />

(Ottaviani and Sorensen, 2001, p.394) or as extreme c<strong>on</strong>currence seeking (Levine and Moreland,<br />

1990; Turner and Pratkanis, 1998). When evaluating organisati<strong>on</strong>s, this leads to problems as those<br />

participating in the evaluati<strong>on</strong>, especially if it is a group based evaluati<strong>on</strong>, may be under pressure to<br />

c<strong>on</strong>form to the group’s views or experience a “a pull towards the group” as described by Asch (1952,<br />

p.483). What the researcher (or practiti<strong>on</strong>er) evaluating the organisati<strong>on</strong> is presented with is<br />

203


John McAvoy, Tadhg Nagle and David Samm<strong>on</strong><br />

c<strong>on</strong>sensus from the participants; this, though, may <strong>on</strong>ly be the illusi<strong>on</strong> of unanimity – if all agree then<br />

it must be true (Argyle, 1989; Manz and Sims, 1982; V<strong>on</strong> Bergen and Kirk, 1978).<br />

The remainder of this paper is organised as follows. The next secti<strong>on</strong> describes <strong>on</strong>e possible method<br />

of deriving real c<strong>on</strong>sensus (and avoiding problems such as groupthink), while also highlighting flaws<br />

with the approach. This is followed by a descripti<strong>on</strong> of a proposed new evaluati<strong>on</strong> method involving a<br />

workshop, and a case study describing the use of this workshop is then described. The results of the<br />

workshop are presented, followed by recommendati<strong>on</strong>s for researchers and practiti<strong>on</strong>ers involved in<br />

evaluating organisati<strong>on</strong>s.<br />

2. Devil’s advocate<br />

Soluti<strong>on</strong>s to the false c<strong>on</strong>sensus seen in groupthink generally involve creating a climate where<br />

decisi<strong>on</strong>s and perspectives are questi<strong>on</strong>ed and critically evaluated, disagreement is encouraged, and<br />

external perspectives sought (cf. Janis, 1972; V<strong>on</strong> Bergen and Kirk, 1978). A proposed method of<br />

doing this is through the use of devil’s advocate.<br />

The use of devil’s advocate, where a member of the team has the task of deliberately opposing or<br />

critiquing the group’s decisi<strong>on</strong>, can provide benefit in creating c<strong>on</strong>fr<strong>on</strong>tati<strong>on</strong> within the group (Thomas,<br />

1988). This technique was used effectively by President Kennedy’s team during the Cuban missile<br />

crisis (Janis, 1972; Thomas, 1988): by using a devil’s advocate (in this case the President’s own<br />

brother Robert), the team avoided errors from an initial superficial analysis, by creating c<strong>on</strong>flict in the<br />

team. The use of devil’s advocate has been described in a variety of research papers, notably<br />

Nemeth and G<strong>on</strong>calo (2004), Schweiger et al. (1989), Herbert and Estes (1977), and Schwenk<br />

(1998). Those who argue for the use of devil’s advocate assume that any decisi<strong>on</strong> or perspective that<br />

can withstand critique is good, where critique, or c<strong>on</strong>flict, reduces the likelihood of a false c<strong>on</strong>sensus<br />

(Cosier, 1981). The use of devil’s advocate has also been shown to have benefits bey<strong>on</strong>d just the<br />

avoidance of groupthink: in decisi<strong>on</strong> making (Hamm<strong>on</strong>d, Keeney and Raiffa, 2006), strategic planning<br />

(Boland, 1984; Mas<strong>on</strong>, 1969), and, specifically for Informati<strong>on</strong> Systems, in ERP projects (Samm<strong>on</strong><br />

and Adam, 2007).<br />

While the benefits of the use of devil’s advocate have been noted above, there is no universal<br />

agreement as to its effectiveness. While Schweiger et al. (1989) argue that the use of devil’s advocate<br />

did not impact <strong>on</strong> a group’s satisfacti<strong>on</strong>; this is not an unc<strong>on</strong>tested argument. For example, Nemeth et<br />

al. (2001) found that antipathy can arise when the devil’s advocate approach is used and that<br />

problems can be created for and within cohesive teams (Nemeth and G<strong>on</strong>calo, 2004). While Herbert<br />

and Estes (1977) argue that this antipathy can be reduced, and Sambamurthy and Poole (1992)<br />

argue that c<strong>on</strong>flict can be beneficial, they also acknowledge that the problems exist and must be dealt<br />

with. Further, Samm<strong>on</strong> and Adam (2007, p.1071) note that “traditi<strong>on</strong>ally, the devil’s advocate<br />

approach, while useful in exposing underlying assumpti<strong>on</strong>s, has a tendency to emphasise the<br />

negative.” This aligns with the argument of Turner and Pratkanis (1998) that soluti<strong>on</strong>s to groupthink<br />

may exacerbate problems in the group if they regard them as intrusi<strong>on</strong>s that questi<strong>on</strong> the group’s<br />

ability to deal with problems.<br />

There are further issues for a researcher or practiti<strong>on</strong>er evaluating an organisati<strong>on</strong>. Typically, the<br />

evaluator will be external to the organisati<strong>on</strong> or team being evaluated. Interventi<strong>on</strong> by an outsider is<br />

not the ideal way of trying to deal with problems such as groupthink as this itself can be part of the<br />

problem. Outsiders’ perspectives are rejected by teams subject to groupthink (Furst, Blackburn and<br />

Rosen, 1999), and an evaluator would be regarded as an outsider. Wastell (1999) talks about a<br />

paranoid view of the world outside of the group where any complaints against the team were incorrect<br />

and unnecessary (Manz and Sims, 1987), with “sloganistic thinking about the immorality of outgroups”<br />

(Oberschal, 1978, p.239). In fact, Janis (1972), the originator of the term groupthink, lists <strong>on</strong>e<br />

symptom of groupthink as advice from outsiders not being sought.<br />

For some<strong>on</strong>e c<strong>on</strong>ducting an evaluati<strong>on</strong>, therefore, there are problems when dealing with teams and<br />

organisati<strong>on</strong>s where false c<strong>on</strong>sensus may be impacting <strong>on</strong> their perspective of reality within the<br />

organisati<strong>on</strong>, and therefore the evaluati<strong>on</strong> of this reality: e.g. if a round table/group discussi<strong>on</strong> as part<br />

of an evaluati<strong>on</strong> was showing symptoms of groupthink. The idea of the devil’s advocate creating<br />

c<strong>on</strong>flict through critique could have benefits, but if d<strong>on</strong>e by an outsider could exacerbate the problem:<br />

ultimately this would not <strong>on</strong>ly lead to an incorrect evaluati<strong>on</strong> but actually worsen the problems that led<br />

to the incorrect evaluati<strong>on</strong>. How then can an evaluator, external to the team or organisati<strong>on</strong> being<br />

204


John McAvoy, Tadhg Nagle and David Samm<strong>on</strong><br />

evaluated, create the necessary critique and c<strong>on</strong>flict required to ensure that the evaluati<strong>on</strong> is a valid<br />

<strong>on</strong>e? The answer appears to be that the critique and c<strong>on</strong>flict must be d<strong>on</strong>e by, and created by, those<br />

being evaluated. The questi<strong>on</strong> though, is how to get a team/organisati<strong>on</strong> to do this. In the next<br />

secti<strong>on</strong>, our agitati<strong>on</strong> workshop is proposed and the method of enabling such a soluti<strong>on</strong> is presented.<br />

3. The “agitati<strong>on</strong> workshop”<br />

The individual c<strong>on</strong>ducting the evaluati<strong>on</strong>, as described above, must facilitate and encourage the<br />

critique and c<strong>on</strong>flict necessary to overcome initial perspectives expressed by the participants, which<br />

may be incorrect and restricted by problems such as groupthink. If the team appear to have a<br />

c<strong>on</strong>sensual perspective of their current situati<strong>on</strong>, and appear to be showing signs of groupthink, then<br />

they must be encouraged to challenge these perspectives: the steps are described below and<br />

illustrated in Figure 1.<br />

In the ‘agitati<strong>on</strong> workshop’ the role of the evaluator is to get the team to challenge their perspectives<br />

and to break apart their c<strong>on</strong>sensual perspectives. Only then can the evaluati<strong>on</strong> be trusted in so far as<br />

it can be assumed to be free of groupthink (and other issues which impact negatively <strong>on</strong> an<br />

individual’s perspective). The term “breaking apart” is deliberately used above as it will involve<br />

c<strong>on</strong>flict. As the evaluator is an outsider, though, the c<strong>on</strong>flict must come from the team and not the<br />

evaluator. The evaluator uses a set of factors which are relevant to the focus of the evaluati<strong>on</strong> (in this<br />

study, the factors used were measures of agility to examine a team’s suitability for Agile methods<br />

adopti<strong>on</strong>: see McAvoy and Samm<strong>on</strong> (2005)). Each workshop participant provides an individual<br />

assessment of the area being evaluated, using a simple binary ‘yes’ (1) or ‘no’ (0) as to the ‘presence’<br />

or ‘absence’ of a factor. Which factors used are not of importance, and are chosen based <strong>on</strong> what is<br />

being evaluated. The critical element of this stage of the workshop is that each participant provides<br />

their own assessment of the factors without any group discussi<strong>on</strong>. If a group discussi<strong>on</strong> <strong>on</strong> the factors<br />

were to take place, it is likely that participants would be influenced by others and the picture<br />

presented to the evaluator would be <strong>on</strong>e that was, what Furst et al. (1999) describe as, an illusi<strong>on</strong> of<br />

c<strong>on</strong>sensus and cohesi<strong>on</strong>.<br />

Once each participant has provided their assessment, these are amalgamated into a single matrix<br />

visualisati<strong>on</strong>. This amalgamated assessment is then presented to the group for discussi<strong>on</strong>. As this<br />

represents the perspectives of the participants, the possibility is removed of the group feeling that it<br />

has been influenced by the ‘incorrect and unnecessary’ (Manz and Sims, 1987) or ‘immoral’<br />

(Oberschal, 1978) views of the outsider – in this case, the evaluator. The evaluator can now lead a<br />

discussi<strong>on</strong> based <strong>on</strong> the matrix (see table 2 for a sample single matrix visualisati<strong>on</strong>). To facilitate<br />

discussi<strong>on</strong>, and to further ensure that it is those being evaluated (as opposed to the evaluator) who<br />

critique their perspectives, the evaluator needs to c<strong>on</strong>centrate <strong>on</strong> some elements of the matrix to<br />

promote discussi<strong>on</strong> (and/or c<strong>on</strong>flict). The factors where there is disagreement between the<br />

participants need to be highlighted in order to promote discussi<strong>on</strong> <strong>on</strong> these factors: this is especially<br />

relevant where there had been a c<strong>on</strong>sensual perspective <strong>on</strong> such factors before the workshop (i.e. in<br />

prior round table or group discussi<strong>on</strong>s). It is the disagreements that will provide the evaluator with the<br />

most opportunity to facilitate the critique/c<strong>on</strong>flict necessary to generate a true picture of the reality<br />

under investigati<strong>on</strong>. Within the single matrix visualisati<strong>on</strong>, a simple count of the 1’s and 0’s will suffice<br />

to show workshop participants that there is no universal agreement <strong>on</strong> these factors.<br />

The discussi<strong>on</strong>s and critique of the differences uncovered is still a team discussi<strong>on</strong> as opposed to a<br />

critique by the evaluator (an outsider). As such, the participants are less likely to reject the different<br />

perspectives that they are c<strong>on</strong>fr<strong>on</strong>ted with than if it was an outsider trying to give advice (as per Janis,<br />

1972). Whereas groupthink creates a false c<strong>on</strong>sensus, the difference in perspectives between<br />

participants cannot be ignored or easily rec<strong>on</strong>ciled to a single (false) c<strong>on</strong>sensual perspective:<br />

therefore, this is working against the pressure to c<strong>on</strong>form to the group’s perspective (as per Asch,<br />

1952). Discussi<strong>on</strong>s now take place where the workshop participants must c<strong>on</strong>fr<strong>on</strong>t their differences<br />

and critique the differentiated perspectives that are being presented. Again, the c<strong>on</strong>flict and critique is<br />

based <strong>on</strong> the perspectives expressed by the participants through their assessments (without the<br />

potential for the ‘corrupting influence’ of outsiders). The term ‘agitati<strong>on</strong> workshop’ comes from the<br />

premise that the (group influenced) collective c<strong>on</strong>sensual perspective has been agitated, through the<br />

use of the single matrix visualisati<strong>on</strong>, to the point that there are now differing perspectives. A<br />

c<strong>on</strong>sensual perspective no l<strong>on</strong>ger exists and a new perspective must be created by the participants<br />

though discussi<strong>on</strong> and critique of the differing perspectives. In the next secti<strong>on</strong>, a case study is<br />

205


John McAvoy, Tadhg Nagle and David Samm<strong>on</strong><br />

presented showing the ‘agitati<strong>on</strong> workshop’ in practice and the benefits that it brought to an evaluati<strong>on</strong><br />

of a global software development organisati<strong>on</strong>.<br />

Figure 1: ‘Agitati<strong>on</strong> workshop’ protocol<br />

4. The “agitati<strong>on</strong> workshop” in practice<br />

The objective of this exploratory study is to determine the feasibility and potential benefits of the<br />

‘agitati<strong>on</strong> workshop’: for this case study research, Texunatech was the organisati<strong>on</strong> in which the<br />

workshop was run. TexunaTech has established itself in the global market as a trusted service<br />

provider of web-based data management applicati<strong>on</strong>s, serving a range of government, healthcare and<br />

private sector organisati<strong>on</strong>s. Coupling geographical locati<strong>on</strong> with specific segments of the ISD<br />

lifecycle, the organisati<strong>on</strong>s structure is defined as follows: (i) L<strong>on</strong>d<strong>on</strong> (UK) – incorporates business<br />

analysis, project management and business development, (ii) Cork (Ireland) – incorporates call centre<br />

operati<strong>on</strong>s, after sales service and first line support, (ii) Moscow (Russia) - incorporates software<br />

development, technology infrastructure maintenance and software testing. The research objective<br />

was in line with the CEO’s requirement, emerging from a str<strong>on</strong>g organisati<strong>on</strong>al necessity, for an<br />

external analysis and evaluati<strong>on</strong> of TexunaTech’s ISD lifecycle and his belief that it could be more<br />

efficient. This was the point of departure for the three pers<strong>on</strong> research team from which the case<br />

study research protocol was developed (see Table1 below).<br />

Table 1: Case study research protocol (after: Kelliher (2005))<br />

Research Activity Descripti<strong>on</strong><br />

Objective To determine the feasibility and potential benefits of the agitati<strong>on</strong> workshop<br />

Approach Case study<br />

Motivati<strong>on</strong> CEO’s desire for an evaluati<strong>on</strong> of ISD lifecycle<br />

Case Selecti<strong>on</strong> A software development organisati<strong>on</strong> where the CEO sought an evaluati<strong>on</strong> of the<br />

Process<br />

organisati<strong>on</strong>’s ways of working<br />

Case Access A unique openness to share informati<strong>on</strong> and a willingness to make pers<strong>on</strong>nel available<br />

for the research, to the extent that operati<strong>on</strong>s were suspended for three days to enable<br />

workshops to be carried out.<br />

Instrument The research team (3 researchers)<br />

Boundary Device ISD lifecycle<br />

Data Gathering Round table discussi<strong>on</strong>s, <strong>on</strong>-site agitati<strong>on</strong> workshops, and group based interviews<br />

Techniques<br />

Data Analysis The agitati<strong>on</strong> workshop single matrix visualisati<strong>on</strong>is used to present and analyse data<br />

Techniques<br />

206


John McAvoy, Tadhg Nagle and David Samm<strong>on</strong><br />

The evaluati<strong>on</strong> commenced with round table discussi<strong>on</strong>s with the various groups in the organisati<strong>on</strong>.<br />

The workshops took place in both the Cork office and the Moscow office, with L<strong>on</strong>d<strong>on</strong> staff travelling<br />

to <strong>on</strong>e or other of the locati<strong>on</strong>s. In these round table discussi<strong>on</strong>s, the different teams within<br />

TexunaTech were asked to provide their perspectives of TexunaTech’s software development<br />

process. It was during these round table discussi<strong>on</strong>s that the researchers noted that symptoms of<br />

groupthink appeared to be impacting <strong>on</strong> the perspectives expressed. The symptoms of groupthink<br />

(from Janis, 1972) that were noted were a pressure to c<strong>on</strong>form to the group’s views (where there<br />

appeared to be a ‘team perspective’ and ‘team answer’ rather than individual perspective) and a<br />

stereotyping of outsiders (where other groups were described negatively and collectively i.e. ‘the<br />

developers d<strong>on</strong>’t take ownership’). This subjective opini<strong>on</strong> of the researchers led to c<strong>on</strong>cerns that an<br />

evaluati<strong>on</strong> based <strong>on</strong> the perspectives expressed might not be accurate as it appeared to be corrupted<br />

by norming and cohesive power of the team. Because of this, it was necessary to ‘explode’ or break<br />

apart the c<strong>on</strong>sensual perspective in order to get an accurate portrayal of each individuals ‘real’<br />

perspective.<br />

For the ‘agitati<strong>on</strong> workshop’ itself, using the protocol described in Figure 1, each participant performed<br />

an individual assessment of the organisati<strong>on</strong>’s suitability for Agile. As can be seen in the single matrix<br />

visualisati<strong>on</strong> in Table 2, there was a large variati<strong>on</strong> in assessments between the participants: due to<br />

space restricti<strong>on</strong>s, <strong>on</strong>ly a porti<strong>on</strong> of <strong>on</strong>e of the workshop assessments is presented, and is reflective<br />

of the other assessments.<br />

Table 2: A sample of the participants differing perspectives of the Agile factors<br />

When the single matrix visualisati<strong>on</strong> was presented to the group, and it became clear to the workshop<br />

participants that, rather than having a c<strong>on</strong>sensual perspective, there were marked differences in<br />

perspectives within the group. They now realised that there were areas where the group did not have<br />

c<strong>on</strong>sensus and that these issues needed to be addressed: additi<strong>on</strong>ally, the team had originally been<br />

in agreement <strong>on</strong> many of these factors before the workshop (during the round table discussi<strong>on</strong>s).<br />

From this point <strong>on</strong>, there was a noticeable increase in discussi<strong>on</strong>, to the extent that a degree of<br />

c<strong>on</strong>flict entered the discussi<strong>on</strong>s. In effect, the group had moved from presenting a c<strong>on</strong>sensual<br />

perspective of the organisati<strong>on</strong> to a situati<strong>on</strong> where the group was discussing and disagreeing with<br />

each other as to the reality within the organisati<strong>on</strong>. It was also noted that the group were more open<br />

to accepting that fault existed within their own team (geographic locati<strong>on</strong>) as opposed to with<br />

outsiders (other teams).<br />

5. Evaluating the evaluati<strong>on</strong><br />

For the evaluator, the questi<strong>on</strong> arises as to whether the new percepti<strong>on</strong>s of the participants in the<br />

workshop are now valid: are they still corrupted by the norming and cohesive power of the group (e.g.<br />

is groupthink still influencing the results?) There are two methods of determining whether the<br />

evaluati<strong>on</strong> is now free of corrupting influences from the group. The first is subjective while the sec<strong>on</strong>d<br />

is a more quantifiable determinati<strong>on</strong>. In the first case, the evaluator can see for themselves whether<br />

the perspectives of the workshop participants are sufficiently different as to create a debate and<br />

critique. In TexunaTech, it was clear that the participants were no l<strong>on</strong>ger expressing the ‘group’s<br />

perspective’; rather they were specifically expressing their own perspective. The level of discussi<strong>on</strong><br />

and disagreement am<strong>on</strong>g participants showed that there were differences between the participants<br />

that had not been expressed before the ‘agitati<strong>on</strong> workshop’ in the round table discussi<strong>on</strong>s. As an<br />

example, in the round table discussi<strong>on</strong>s before the workshop, business analysts (BAs) had all agreed<br />

that they were good at dealing with customers. After seeing the single matrix visualisati<strong>on</strong> (customer<br />

involvement (CI) factor in Table 2), they came to realise that, despite the previous agreement,<br />

customer involvement was problematic: several BAs then commented that they “felt helpless” when<br />

dealing with the customer. This would be highly unlikely to have occurred if groupthink was<br />

influencing them: if groupthink was influencing the evaluati<strong>on</strong> then these differences in perspective<br />

would be absent from the discussi<strong>on</strong>s. Additi<strong>on</strong>ally, in the TexunaTech case, it became very clear that<br />

individuals no l<strong>on</strong>ger solely blamed other groups for problems and were more willing to blame their<br />

207


John McAvoy, Tadhg Nagle and David Samm<strong>on</strong><br />

own group and themselves. For example, the developers had originally blamed the BAs for the late<br />

clarificati<strong>on</strong> of requirements. After the workshop, the developers were more understanding of the<br />

difficulties involved for the BAs and also accepted that part of the fault existed with themselves for not<br />

addressing the issue. For years, the developers handled late requirements without adequately<br />

explaining the impact to the BAs: they now accepted that they should have challenged the status quo.<br />

The sec<strong>on</strong>d method involved the use of a mindfulness instrument to evaluate the state of the<br />

organisati<strong>on</strong> before and after the ‘agitiati<strong>on</strong> workshop’. Mindfulness is a process that promotes ‘selfregulati<strong>on</strong><br />

of attenti<strong>on</strong>’ and ‘a posture of acceptance’ in individuals (Hayes and Shenk, 2004, p. 249)<br />

providing better process awareness and str<strong>on</strong>ger accountability, and is a feature of High Reliability<br />

Organisati<strong>on</strong>s (Weick, Sutcliffe and Obstfeld, 1999). For more details <strong>on</strong> mindfulness assessments<br />

and the comp<strong>on</strong>ents of mindfulness see Mu and Butler (2009) and Nagle et al. (2011).<br />

Mindfulness assessments were c<strong>on</strong>ducted by each participant, and standard deviati<strong>on</strong> in mindfulness<br />

assessments was used to quantify the degree of difference in perspectives, and how much the<br />

differences have increased/decreased between the pre and post ‘agitati<strong>on</strong> workshop’. These<br />

Mindfulness measures were taken before and after the workshop to determine the impact of the<br />

‘agitati<strong>on</strong> workshop’, where mindfulness was a measure of the workshop effectiveness; it was not the<br />

focus of the evaluati<strong>on</strong>. The differences in assessments were noticeable, with the group lowering their<br />

evaluati<strong>on</strong> of the organisati<strong>on</strong> (and in a minor number of cases raising it). Overall, the difference in<br />

assessment (the assessment returns a numeric assessment of mindfulness) was negative: e.g. <strong>on</strong>e<br />

element of the assessment dropping by 24% between pre and post workshop. If the standard<br />

deviati<strong>on</strong> of the participants’ mindfulness assessments has increased, then the level of disagreement<br />

has also increased. In some cases the standard deviati<strong>on</strong> may decrease for some parts of the<br />

evaluati<strong>on</strong>: in these cases it shows that any debate and discussi<strong>on</strong> has actually led to more<br />

agreement within the team. This would not be unexpected, but the evaluator would need to be<br />

cautious with the perspectives expressed if all the values came closer to agreement (as this implies<br />

that normative pressures, such as groupthink, may be corrupting the discussi<strong>on</strong>s). For the<br />

TexunaTech case, some values had a decrease in standard deviati<strong>on</strong> (more agreement between<br />

individuals) but the majority of standard deviati<strong>on</strong>s increased (more disagreement within the team). It<br />

is argued herein that the increase in standard deviati<strong>on</strong> shows that the individual values<br />

(perspectives) are less likely to have been corrupted by normative pressure, and thus are more likely<br />

to be a more accurate reflecti<strong>on</strong> of reality. This more accurate versi<strong>on</strong> of reality can now be used, with<br />

more c<strong>on</strong>fidence, in the evaluati<strong>on</strong> of the team/organisati<strong>on</strong>. Table 3 illustrates the differences in<br />

standard deviati<strong>on</strong> in TexunaTech, between the individual perspectives before the ‘agitati<strong>on</strong><br />

workshop’ (represented as Pre STDEV) and the perspectives after the ‘agitati<strong>on</strong> workshop’<br />

(represented as Post STDEV).<br />

Table 3: Measuring differences in perspectives caused by agitati<strong>on</strong> workshop<br />

Reluctance to simplify<br />

interpretati<strong>on</strong>s Preoccupati<strong>on</strong> with failure Sensitivity to operati<strong>on</strong>s<br />

MDRS1 MDRS2 MDRS3 MDRS4 MDPF1 MDPF2 MDPF3 MDSO1 MDSO2 MDSO3 MDSO4 MDSO5<br />

Pre STDEV 1.191 1.095 1.044 1.834 0.786 1.342 1.272 0.505 1.036 0.688 1.000 0.647<br />

Post STDEV 1.598 1.246 0.991 1.506 1.389 1.506 0.835 1.414 1.642 1.669 1.309 1.356<br />

Difference 0.407 0.151 ‐0.053 ‐0.328 0.603 0.164 ‐0.437 0.909 0.606 0.981 0.309 0.709<br />

Commitment to Resilience Deference to expertise<br />

MDCR1 MDCR2 MDCR3 MDCR4 MDEX1 MDEX2 MDEX3 MDEX4<br />

Pre STDEV 0.647 0.674 1.567 0.944 0.522 0.688 0.944 0.786<br />

Post STDEV 0.535 1.302 1.165 1.356 1.414 0.835 1.553 1.188<br />

Difference ‐0.112 0.628 ‐0.402 0.412 0.892 0.147 0.609 0.402<br />

To determine the effectiveness of the ‘agitati<strong>on</strong> workshop’, there are two relevant points to be taken<br />

from the differences in the standard deviati<strong>on</strong>s above. Firstly, the count of values that showed an<br />

increase in standard deviati<strong>on</strong> was four times greater than the count with a decrease in standard<br />

deviati<strong>on</strong>. This clearly shows that perspectives are much more differentiated across values, post<br />

workshop. Sec<strong>on</strong>dly, the size of the increase versus the size of the decrease is noticeable. As the<br />

mindfulness measurement values ranged from 1 to 7, a standard deviati<strong>on</strong> change of 0.5 or more was<br />

regarded as significant. Nearly half of the increases in standard deviati<strong>on</strong> were greater than this<br />

value, showing that there was a significant increase in the differences of perspectives between the<br />

workshop participants; there were no decreases in standard deviati<strong>on</strong> (coming together of<br />

perspectives) of significance. Even ignoring the use of a significant value (0.5 in this case), the<br />

208


John McAvoy, Tadhg Nagle and David Samm<strong>on</strong><br />

mathematical sum of all changes in standard deviati<strong>on</strong> is positive (more disagreement than<br />

agreement) and the average increase in difference was twice the size of the average decrease in<br />

difference. Taking all of these numerical calculati<strong>on</strong>s together, it is clear that the workshop created<br />

more differences in perspectives between the participants in their assessment of the organisati<strong>on</strong>s<br />

software development process.<br />

6. C<strong>on</strong>clusi<strong>on</strong>s<br />

The protocol for the ‘agitati<strong>on</strong> workshop’ dem<strong>on</strong>strated the desired effect: ‘breaking up’ the comm<strong>on</strong><br />

c<strong>on</strong>sensual perspective, which was influenced by groupthink, and giving the participants the<br />

opportunity to create a new (more accurate) percepti<strong>on</strong> of reality. It is clear, from the example in the<br />

case above, that the values taken after the workshop are free (or more free) of corrupting normative<br />

pressures such as groupthink. This should give the evaluator more c<strong>on</strong>fidence in the perspectives<br />

expressed by the participants after the workshop. Ultimately this should ensure a more accurate<br />

representati<strong>on</strong> of the reality being evaluated and the problems (or positive aspects) seen.<br />

It is not relevant to the researcher whether the new reality represented by the participants is a more<br />

positive perspective of the team/organisati<strong>on</strong> or more negative perspective. What is important is that<br />

the evaluator has more c<strong>on</strong>fidence in the picture of reality presented by those being evaluated i.e. the<br />

evaluator has, through the ‘agitati<strong>on</strong> workshop’, minimised the level of normative group corrupti<strong>on</strong> of<br />

the participants individual percepti<strong>on</strong>s. This can <strong>on</strong>ly lead to more accurate evaluati<strong>on</strong>s.<br />

References<br />

Argyle, M. (1989) The social psychology of work, Penguin Books, Middlesex, UK.<br />

Argyris, C. and Sch<strong>on</strong>, D. (1978) Organizati<strong>on</strong>al learning: A theory of acti<strong>on</strong> perspective, Addis<strong>on</strong>-Wesley, MA,<br />

USA.<br />

Asch, S. (1952) Social Psychology, Prentice-Hall, NJ, USA.<br />

Bjorn, P. and Morten, H. (2005) Proactive behaviour may lead to failure in virtual project-based collaborative<br />

learning, Internati<strong>on</strong>al ACM SIGGROUP c<strong>on</strong>ference <strong>on</strong> Supporting group work (Eds, Pendergast, M.,<br />

Schmidt, K., Mark, G. and Ackerman, M.) ACM, Florida.<br />

Boland, R. (1984) Sense-making of accounting data as a technique of organisati<strong>on</strong>al diagnosis, <strong>Management</strong><br />

Science, 30 (7), pp. 868-882.<br />

Cosier, R. (1981) Dialectical inquiry is strategic planning: a case of premature acceptance?, The Academy of<br />

<strong>Management</strong> Review, 6 (4), pp. 643–648.<br />

Esser, J. (1998) Alive and well after 25 years: A review of groupthink research, Organizati<strong>on</strong>al Behaviour and<br />

Human Decisi<strong>on</strong> Processes, 73 (2/3), pp. 116-141.<br />

Furst, S., Blackburn, R. and Rosen, B. (1999) Virtual team effectiveness: a proposed research agenda,<br />

Informati<strong>on</strong> Systems Journal, 9 (4), pp. 249-269.<br />

Hamm<strong>on</strong>d, J., Keeney, R. and Raiffa, H. (2006) The hidden traps in decisi<strong>on</strong> making, Harvard Business Review,<br />

Vol. January, pp. 118-126.<br />

Hayes, S. and Shenk, C. (2004) Operati<strong>on</strong>alizing mindfulness without unnecessary attachments, Clinical<br />

psychology: Science and practice, 11 (3), pp. 249-254.<br />

Herbert, T. and Estes, R. (1977) Improving executive decisi<strong>on</strong>s by formalizing dissent: the corporate devil’s<br />

advocate, The Academy of <strong>Management</strong> Review, 2 (4), pp. 662–667.<br />

Janis, I. (1972) Victims of groupthink, Hought<strong>on</strong> Mifflin Company, MA, USA.<br />

Kelliher, F. (2005) Interpretivism and the pursuit of research legitimisati<strong>on</strong>: An integrated approach to single case<br />

design, The Electr<strong>on</strong>ic Journal of Business Research Methodology, 3 (2), pp. 123-132.<br />

Leana, C. (1985) A partial test of Janis’ groupthink model: effects of group cohesiveness and leader behaviour <strong>on</strong><br />

defective decisi<strong>on</strong> making, Journal of <strong>Management</strong>, 11 (1), pp. 5-17.<br />

Levine, J. and Moreland, R. (1990) Progress in small group research, Annual Review of Psychology, 41 (1), pp.<br />

585-634.<br />

Love, K. (2000) The regulati<strong>on</strong> of argumentative reas<strong>on</strong>ing in pedagogic discourse, Discourse Studies, 2 (4), pp.<br />

420-451.<br />

Manz, C. and Sims, H. (1982) The potential for ‘groupthink’ in aut<strong>on</strong>omous work groups, Human Relati<strong>on</strong>s, 35<br />

(9), pp. 773-784.<br />

Manz, C. and Sims, H. (1987) Leading workers to lead themselves: The external leadership of self-managing<br />

work teams, Administrative Science Quarterly, 32 (1), pp. 106-128.<br />

Mas<strong>on</strong>, R. (1969) A dialectical approach to strategic planning, <strong>Management</strong> Science, 15 (8), pp. 403-414.<br />

McAvoy, J. and Butler, T. (2009) The role of project management in ineffective decisi<strong>on</strong> making within Agile<br />

software development projects, <str<strong>on</strong>g>European</str<strong>on</strong>g> Journal of Informati<strong>on</strong> Systems, 18 (4), pp. 372-383.<br />

McAvoy, J. and Samm<strong>on</strong>, D. (2005) Agile methodology adopti<strong>on</strong> decisi<strong>on</strong>s: An innovative approach to teaching<br />

and learning, Journal of Informati<strong>on</strong> Systems Educati<strong>on</strong>, 16 (4), pp. 409-420.<br />

McMah<strong>on</strong>, J. (2003) 5 less<strong>on</strong>s from transiti<strong>on</strong>ing to eXtreme Programming, C<strong>on</strong>trol Engineering, Vol. 50, pp. 59-<br />

60.<br />

209


John McAvoy, Tadhg Nagle and David Samm<strong>on</strong><br />

Mu, E. and Butler, B. (2009) The Assessment of Organizati<strong>on</strong>al Mindfulness Processes for the Effective<br />

Assimilati<strong>on</strong> of IT Innovati<strong>on</strong>s, Journal of Decisi<strong>on</strong> Systems, 18 (1), pp. 27-51.<br />

Nagle, T., McAvoy, J. and Samm<strong>on</strong>, D. (2011) Utilising Mindfulness to analyse Agile global software<br />

development, <str<strong>on</strong>g>European</str<strong>on</strong>g> <str<strong>on</strong>g>C<strong>on</strong>ference</str<strong>on</strong>g> <strong>on</strong> Informati<strong>on</strong> Systems, Helsinki.<br />

Nemeth, C., Brown, K. and Rogers, J. (2001) Devil’s advocacy versus authentic dissent: Stimulating quantity and<br />

quality, <str<strong>on</strong>g>European</str<strong>on</strong>g> Journal of Social Psychology, 31 (6), pp. 707-720.<br />

Nemeth, C. and G<strong>on</strong>calo, J. (2004) Influence and persuasi<strong>on</strong> in small groups, In Persuasi<strong>on</strong>: Psychological<br />

insights and perspectives(Eds, Shavitt, S. and Brock, T.) Allyn and Bac<strong>on</strong>, MA, USA, pp. 171-194.<br />

Oberschal, A. (1978) Theories of social c<strong>on</strong>flict, Annual review of sociology, 4 291-315.<br />

Ottaviani, M. and Sorensen, P. (2001) Informati<strong>on</strong> aggregati<strong>on</strong> in debate: Who should speak first, Journal of<br />

Public Ec<strong>on</strong>omics, 81 (3), pp. 393-421.<br />

Pfeffer, J. and Sutt<strong>on</strong>, R. (2000) The Knowing Doing Gap: How Smart Companies Turn Knowledge Into Acti<strong>on</strong>,<br />

Harvard Business School Press, MA, USA.<br />

Richards<strong>on</strong>, V. (2003) C<strong>on</strong>structivist pedagogy, Teachers College Record, 105 (9), pp. 1623-1640.<br />

Sambamurthy, V. and Poole, M. (1992) The effects of variati<strong>on</strong>s in capabilities of GDSS designs <strong>on</strong> management<br />

of cognitive c<strong>on</strong>flict in groups, Informati<strong>on</strong> Systems Research, 3 (3), pp. 224–251.<br />

Samm<strong>on</strong>, D. and Adam, F. (2007) An Extended Model of Decisi<strong>on</strong> Making for a Mindful Approach to IT<br />

Innovati<strong>on</strong>s (Enterprise-Wide ERP Project Implementati<strong>on</strong>), <str<strong>on</strong>g>European</str<strong>on</strong>g> <str<strong>on</strong>g>C<strong>on</strong>ference</str<strong>on</strong>g> <strong>on</strong> Informati<strong>on</strong><br />

Systems(Eds, Osterle, H., Schelp, J. and Winter, R.) St Gallens, Switzerland, pp. 1064-1076.<br />

Schweiger, D., Sandberg, W. and Rechner, P. (1989) Experiential effects of dialectical inquiry, devil’s advocacy,<br />

and c<strong>on</strong>sensus approaches to strategic decisi<strong>on</strong> making, Academy of <strong>Management</strong> Journal, 32 (4), pp.<br />

745–772.<br />

Schwenk, C. (1998) Effects of devil’s advocacy <strong>on</strong> escalating commitment, Human Relati<strong>on</strong>s, 41 (10), pp. 769–<br />

782.<br />

Thomas, H. (1988) Policy dialogue in strategic planning: Talking our way through ambiguity and change, In<br />

Managing ambiguity and change(Eds, P<strong>on</strong>dy, L., Boland, R. and Thomas, H.) John Wiley and S<strong>on</strong>,<br />

Chichester, UK, pp. 51-77.<br />

Turner, M. and Pratkanis, A. (1998) A social identity maintenance model of groupthink, Organizati<strong>on</strong>al Behaviour<br />

and Human Decisi<strong>on</strong> Processes, 73 (2/3), pp. 210-235.<br />

V<strong>on</strong> Bergen, C. and Kirk, R. (1978) Groupthink: When too many heads spoil the decisi<strong>on</strong>, <strong>Management</strong> Review,<br />

67 (3), pp. 44-49.<br />

Wastell, D. (1999) Learning dysfuncti<strong>on</strong>s in informati<strong>on</strong> systems development: Overcoming the social defenses<br />

with traditi<strong>on</strong>al objects, MIS Quarterly, 23 (4), pp. 581-600.<br />

Weick, K., Sutcliffe, K. and Obstfeld, D. (1999) Organizing for high reliability: Processes of collective mindfulness,<br />

Research in Organizati<strong>on</strong>al Behavior, 21 81-123.<br />

210


Drivers and Challenges for Biometrics in the Financial Services<br />

Karen Neville<br />

Business Informati<strong>on</strong> Systems, University College Cork, Ireland<br />

KarenNeville@UCC.ie<br />

Abstract: Mobile banking and biometrics are currently making profound changes to the Financial Services<br />

landscape. Any<strong>on</strong>e with access to a cell ph<strong>on</strong>e has a place to keep his or her savings without needing a<br />

traditi<strong>on</strong>al bank account. The Mobile payment value chain has various roles, all of which need to be addressed<br />

and managed. There is a potential for mobile operators, security technology organizati<strong>on</strong>s can target FS<br />

organizati<strong>on</strong>s and provide both the technical soluti<strong>on</strong>, business experience and collaborative forum necessary to<br />

solve barriers for banks such as security (biometric identity assurance), regulati<strong>on</strong>s/standards (collaborative<br />

forum) and partnerships. This opportunity for mobile payments has already been verified through preliminary<br />

analysis by blue chip FS organisati<strong>on</strong>s. Additi<strong>on</strong>ally there is the potential of “piggybacking” <strong>on</strong> the nati<strong>on</strong>al ID in<br />

emerging markets to allow payments functi<strong>on</strong>ality. This would allow poorer countries without a FS infrastructure<br />

capability to “leapfrog” to financial inclusi<strong>on</strong>. As The New York Times Magazine noted in a recent cover story, last<br />

year migrants across the globe sent home $300 billi<strong>on</strong>. The potential of tapping this market by Financial Services<br />

and Mobile operators is enormous. In additi<strong>on</strong> to the risk of an un-standardised market, banks are at risk of not<br />

facilitating such a market in developing countries from new competitors siph<strong>on</strong>ing potential customers. Therefore<br />

the remittance and unbanked markets are where m-Banking will be world-changing. The objective of this study is<br />

two–pr<strong>on</strong>ged: it is both to determine and highlight the potential of Biometrics for the Financial Services market<br />

Keywords: biometrics, financial services, banks, regulati<strong>on</strong><br />

1. Introducti<strong>on</strong><br />

Due to the risk of an un-standardised market, banks are at risk of not facilitating such a market in the<br />

US, Europe and developing countries from new competitors siph<strong>on</strong>ing potential customers. Mobile<br />

banking is c<strong>on</strong>sidered a lucrative opti<strong>on</strong> for the majority of the banks interviewed with the<br />

acknowledgement that partners will be needed. To ensure security as well as reliability and<br />

trustworthiness in electr<strong>on</strong>ic transmissi<strong>on</strong> of data, certain issues need to be taken into account and<br />

resolved, including: authenticati<strong>on</strong>, integrity, n<strong>on</strong>-repudiati<strong>on</strong> and c<strong>on</strong>fidentiality. To this end, a<br />

number of technologies and paradigms can be brought to bear <strong>on</strong> activities such that risk factors are<br />

reduced to an acceptable level, problems can be tracked seamlessly and business c<strong>on</strong>tinuity<br />

principles are upheld. Am<strong>on</strong>gst the technologies that are currently being investigated, this report<br />

presents a business case for biometrics, specifically biometrics in acti<strong>on</strong>, identity challenges and case<br />

studies of current deployments. In 2001 MIT identified biometrics as <strong>on</strong>e of the top 5 technologies “…<br />

to watch out for…” and in 2012 this foresight has been proven accurate with the commercialisati<strong>on</strong> of<br />

the technology into everyday life. The American, <str<strong>on</strong>g>European</str<strong>on</strong>g> Uni<strong>on</strong>, Australian and Japanese<br />

governments have all adopted and introduced the technology to nearly 250 milli<strong>on</strong> unique users as<br />

entry safeguards.<br />

Biometrics enables the identificati<strong>on</strong> of individuals by specific biological traits thus assuring<br />

transacti<strong>on</strong>s of varied proporti<strong>on</strong>s. This assurance has propelled its acceptance by the public offering<br />

unparalleled security advantages, c<strong>on</strong>venience as you are your password, robustness in meeting<br />

whichever challenge is posed and critical to any deployment. The technology is not merely an ‘add <strong>on</strong>’<br />

to existing security systems it is essentially integrated into business processes allowing a seemliness<br />

secure layer to verify the different stakeholders in large scale deployments such as missi<strong>on</strong> critical<br />

security systems. This paper attests to the value of the technology in assuring security to customers,<br />

merchants, c<strong>on</strong>tent providers, financial instituti<strong>on</strong>s, governments and mobile operators to name but a<br />

few.<br />

2. Biometrics<br />

Biometrics facilitates the automatic authenticati<strong>on</strong> of a living pers<strong>on</strong> based <strong>on</strong> his/her unique<br />

physiological or behavioural characteristics. Comm<strong>on</strong> physical biometrics include: fingerprints, hand<br />

or palm geometry, and retina, iris, ear shape or facial characteristics. Behavioural characters include<br />

signature, voice, keystroke pattern, and gait. Of this class of biometrics, technologies for signature<br />

and voice are the most developed. There are two main reas<strong>on</strong>s that biometrics are used in security<br />

systems; (1) to verify or (2) to identify users. Verificati<strong>on</strong> involves c<strong>on</strong>firming or denying a pers<strong>on</strong>'s<br />

claimed identity. In identificati<strong>on</strong>, <strong>on</strong>e has to establish a pers<strong>on</strong>'s identity, which tends to be the more<br />

difficult of the two uses because a system must search a database of enrolled users to find a match,<br />

211


Karen Neville<br />

which is a <strong>on</strong>e-to-many search whereas verificati<strong>on</strong> is a <strong>on</strong>e-to-<strong>on</strong>e search. Biometrics, as a form of<br />

identificati<strong>on</strong>, is preferred over traditi<strong>on</strong>al methods involving passwords and PIN numbers as the<br />

pers<strong>on</strong> to be identified is required to be physically present at the point-of-identificati<strong>on</strong>. Biometrics<br />

removes this disadvantage and therefore the cost incurred in being physically present for verificati<strong>on</strong>.<br />

In additi<strong>on</strong> to this c<strong>on</strong>venience customers no l<strong>on</strong>ger have to remember password/s or carry a token.<br />

This c<strong>on</strong>trol is utilised in every industry as serious identity challenges and poor <strong>on</strong>line authenticati<strong>on</strong><br />

is threatening an invaluable channel to customers. Additi<strong>on</strong>ally insider fraud is enabled through poor<br />

access c<strong>on</strong>trol. The drive to protect identity has led to increased security measures; yet restrictive<br />

security measures can reduce efficiency for an organisati<strong>on</strong> and can be an inc<strong>on</strong>venience to<br />

customers. However, identity management properly developed and executed can mitigate risk, enable<br />

transacti<strong>on</strong>s, and act as a source of competitive differentiati<strong>on</strong> and therefore advantage. Central to<br />

achieving this is combining security and c<strong>on</strong>venience. Identity management can be moved from a<br />

“must do” to a “must have”. In fact identity management is cited in the Deloitte Global 2007 Security<br />

Survey as <strong>on</strong>e the primary security issues for organisati<strong>on</strong>s to solve (www2). Biometric technologies<br />

provide identity management soluti<strong>on</strong>s through the automatic authenticati<strong>on</strong> of individuals. Biometrics<br />

has an inherent ability to assure security, provide c<strong>on</strong>venience and mobility through their uniqueness,<br />

universality, permanence and acceptability. Biometrics enable authenticati<strong>on</strong> for the world’s major<br />

border security systems such as the US and Japanese VISIT authenticati<strong>on</strong> systems. These missi<strong>on</strong><br />

critical systems are large scale deployments of the technology proving its scalability and robustness in<br />

meeting security threats. Ultimately the technology can counteract whichever challenges have driven<br />

its acceptance.<br />

2.1 Drivers and challenges for biometrics<br />

Incidents such as <strong>on</strong>line attacks, insider fraud and data breaches have damaged instituti<strong>on</strong>s’ brands<br />

and reduced c<strong>on</strong>fidence in for example <strong>on</strong>line banking. Attack trends have grown more sophisticated<br />

as banks have sought to counter fraud with improved security and customer educati<strong>on</strong>. The<br />

perpetrators of cybercrime have become organized; far from it being a case of keeping <strong>on</strong>e step,<br />

ahead of criminals, many companies are struggling to keep pace. Figure 1 illustrates examples of<br />

some of well-known incident/threats and ultimately drivers for the adopti<strong>on</strong> of a secure identity<br />

management/biometric technologies. The rampant spread of identity theft, as well as the above<br />

challenges, has been significant drivers in the accelerati<strong>on</strong> of biometrics adopti<strong>on</strong>. C<strong>on</strong>sumers are<br />

welcoming the c<strong>on</strong>venience, privacy and security advantages associated with enrolling their<br />

biometrics with merchants and organizati<strong>on</strong>s.<br />

Figure 1: Biometric drivers and challenges<br />

212


2.2 Biometrics industry in acti<strong>on</strong><br />

Karen Neville<br />

The biometrics industry has evolved into the most secure and c<strong>on</strong>venient approach to identity<br />

management as a biometric cannot be borrowed, stolen, or forgotten, and forging is practically<br />

impossible. Token-based methods of identificati<strong>on</strong> like passports and driver's licenses, may be forged,<br />

stolen, or lost which is why biometrics are so important for increased security. Complex passwords<br />

are easy to forget, while simple passwords can be easily guessed by unauthorised people. In large<br />

scale missi<strong>on</strong> critical deployments across the world biometrics have assured security and<br />

c<strong>on</strong>venience. Some possible applicati<strong>on</strong>s and advantages of biometrics include: (1) Physical access;<br />

(2) Virtual access; (3) E-commerce Applicati<strong>on</strong>s (4) M-Commerce Applicati<strong>on</strong>s (5) Covert surveillance<br />

and (6) Smart-cards.<br />

Physical access<br />

The primary applicati<strong>on</strong> of biometrics is to c<strong>on</strong>trol access to secure locati<strong>on</strong>s such as rooms or<br />

buildings. Unlike photo identificati<strong>on</strong> cards, which a security guard must verify, biometrics permits<br />

unmanned access c<strong>on</strong>trol. Disney World for example uses a fingerprint scanner to verify seas<strong>on</strong>-pass<br />

holders entering the theme park.<br />

Virtual access<br />

Biometrics can increase a company's ability to protect its data by implementing a more secure key<br />

than a password and employees are freed from the burdens of password-based log<strong>on</strong>s to operating<br />

systems and applicati<strong>on</strong>s. Biometrics carries this out automatically and forgotten passwords are a<br />

thing of the past which not <strong>on</strong>ly reduces the costs for hotline/system administrati<strong>on</strong> in large-scale<br />

envir<strong>on</strong>ments, but also increases productivity because employees no l<strong>on</strong>ger need to sit around<br />

waiting for the administrator to reset their forgotten password. Users can log<strong>on</strong> to a Windows<br />

operating system for example, by using a biometrically-activated smart-card.<br />

E-commerce Applicati<strong>on</strong>s<br />

E-commerce developers are exploring the use of biometrics and smart-cards for the purpose of<br />

verifying a trading party's identity, to lessen fraud and ensure n<strong>on</strong>-repudiati<strong>on</strong>. Point-Of-Sales (POS)<br />

system vendors are working <strong>on</strong> the cardholder verificati<strong>on</strong> method, using smart-cards and biometrics<br />

to replace signature verificati<strong>on</strong>.<br />

M-commerce Applicati<strong>on</strong>s<br />

There are over 3 billi<strong>on</strong> mobile ph<strong>on</strong>e users globally and the use of mobile devices for payments is<br />

steadily increasing. Combining biometrics with mobile ph<strong>on</strong>es for authenticati<strong>on</strong> is an obvious soluti<strong>on</strong><br />

to binding the user to a transacti<strong>on</strong>. Additi<strong>on</strong>ally mobile ph<strong>on</strong>es are equipped with voice capture<br />

capability with the added potential of using inbuilt cameras. The combinati<strong>on</strong> of these two<br />

technologies is changing the payments landscape. Mobile banking is also expected to drive the<br />

utilisati<strong>on</strong> further with numerous banks (Bank of America, Citibank, and America First) offering mobile<br />

banking products (www3) and forming partnerships with Mobile carriers and trusted third parties.<br />

Covert surveillance<br />

One of the more challenging research areas involves using biometrics for covert surveillance. Using<br />

facial and body recogniti<strong>on</strong> technologies, researchers hope to use biometrics to automatically identify<br />

known suspects entering buildings or traversing crowded security areas such as airports.<br />

Biometrics & Smart-cards<br />

A smart-card is a “...credit card sized c<strong>on</strong>venti<strong>on</strong>al plastic card”, c<strong>on</strong>taining an integrated circuit chip<br />

allowing significant amounts of informati<strong>on</strong> to be stored, accessed and processed either <strong>on</strong>line or<br />

offline. The following are the advantages of combining smart-card technology with biometrics: security<br />

(two-factor authenticati<strong>on</strong> which c<strong>on</strong>siderably reduces fraud), memory size (allowing them to carry<br />

biometrics and encrypti<strong>on</strong>), portability (not limited to a particular desktop), c<strong>on</strong>venience (enabling an<br />

effective transiti<strong>on</strong> for Financial Services), multiple applicati<strong>on</strong>s (allows companies to collaborate with<br />

partners), cost savings (by eliminating paper and handling) and micro-charging (a cheap alternative to<br />

cheques).<br />

The combinati<strong>on</strong> of biometrics and smart-cards is a familiar applicati<strong>on</strong> of biometrics. Biometrics offer<br />

simplicity and str<strong>on</strong>g authenticati<strong>on</strong> and has also proven robust and scalable in a range of large scale<br />

deployments. In additi<strong>on</strong> to the applicati<strong>on</strong>s outlined above the integrati<strong>on</strong> of biometrics with smart-<br />

213


Karen Neville<br />

cards has also been identified, in industry, as a c<strong>on</strong>venient transiti<strong>on</strong> for Banks into the biometrics<br />

field or as a legacy issue in the case of existing use.<br />

3. Solving identity issues in financial services<br />

Biometrics has proven its l<strong>on</strong>gevity, scalability, performance and reliability due to its adopti<strong>on</strong> by<br />

governments throughout the world. The financial services have been exploiting the technology and it<br />

is <strong>on</strong>e of the key markets positi<strong>on</strong>ing itself for explosive growth in niche areas such as mobile<br />

banking. Fully integrated biometrically enabled financial service systems that efficiently and<br />

c<strong>on</strong>veniently serve milli<strong>on</strong>s are also evolving. Biometric technology can achieve significant business<br />

process improvement and enhance end user c<strong>on</strong>venience. Current biometrics penetrati<strong>on</strong> in financial<br />

services will c<strong>on</strong>tinue <strong>on</strong> its current path of successful implementati<strong>on</strong>s and c<strong>on</strong>tinued growth. The<br />

development and deployment of fully integrated biometric-enabled financial systems has resulted<br />

from improvements in technology (accuracy, reliability, processing speed, durability, and usability)<br />

despite the dynamics of the financial services market. Numerous deployments have provided<br />

identificati<strong>on</strong> & authenticati<strong>on</strong> soluti<strong>on</strong>s where biometrics dem<strong>on</strong>strated reliability, quantifiable return<br />

<strong>on</strong> investment and operati<strong>on</strong>al performance.<br />

Customer acceptance of biometrics has been significant. This is especially true at the point-of-sale<br />

(POS). Several vendors have taken biometrics directly to c<strong>on</strong>sumers in retail applicati<strong>on</strong>s and much<br />

to the surprise of many industry pundits and privacy advocates alike, c<strong>on</strong>sumers are resp<strong>on</strong>ding with<br />

curiosity and enthusiasm. From iris scan ATMs to finger-scan supermarket checkout, intrigue<br />

outweighs fear. In <strong>on</strong>e early 1998 bank pilot in Texas, 80 percent of customers who could use iris<br />

scan ATMs did. Ninety-five percent of these customers said they were satisfied and 35 percent said<br />

they had opened accounts specifically because of the biometric ATMs. However the primary<br />

obstacles to rapid uptake of biometrics are the cost and complexity of deployments. These issues<br />

include legacy systems integrati<strong>on</strong>, enrolment, infrastructure, database storage, identificati<strong>on</strong> and<br />

authenticati<strong>on</strong> management, interoperability and standards. As with most industry sectors evaluating<br />

biometrics, the most successful financial service pilots and deployments have proven that the benefits<br />

far exceed the costs.<br />

The drive within financial services towards biometrics is based <strong>on</strong> the urgent need to leverage<br />

potential efficiencies of existing IT infrastructure investments particularly for E-commerce and the<br />

Internet while decreasing costs. In order to drive down operati<strong>on</strong> costs, financial service organizati<strong>on</strong>s<br />

must reduce the cost of each customer interacti<strong>on</strong>. Customer self-service is the key and str<strong>on</strong>g<br />

authenticati<strong>on</strong> is essential to this process. As self-service increases, str<strong>on</strong>g authenticati<strong>on</strong> becomes<br />

increasingly more cumbersome for each individual and biometrics as an alternative to pins,<br />

passwords and tokens becomes ever more appealing.<br />

Financial industry analysts suggest that widespread internal use of biometrics for employee-facing<br />

applicati<strong>on</strong>s will likely precede roll-outs of large-scale, customer-facing deployments. It is possible;<br />

however, that effective deployment of biometrics at the point-of-sale (POS) may pressure some<br />

sectors of the industry to deploy customer-facing applicati<strong>on</strong>s so<strong>on</strong>er. The following secti<strong>on</strong> illustrates<br />

some examples of biometrics in acti<strong>on</strong>, particularly emphasising customer facing applicati<strong>on</strong>s in<br />

Financial Services.<br />

3.1 Financial services cases<br />

Established vendors and new entrants have identified niches where biometrics provide a unique<br />

soluti<strong>on</strong>, the following are some examples of current deployments:<br />

Banking group Deploys Voice-verified Password Reset-service<br />

Allied Irish Banks (AIB) has rolled out a VoiceVault's Password Reset Service with initial<br />

implementati<strong>on</strong>s in head office locati<strong>on</strong>s. The bank has enrolled more than 5,700 of its employees to<br />

optimise IT helpdesk resources and reduce costs. Users of AIB IT systems speak a few words into<br />

the VoiceVault-powered system and these are used to biometrically voice-verify the staff member's<br />

identity over the ph<strong>on</strong>e. Once verified, the user's password is automatically reset and spoken back to<br />

them. As well as being simple and quick to use, VoiceVault enhances password security and<br />

substantially reduces the volume of IT helpdesk calls so that support teams can focus <strong>on</strong> other user<br />

issues. Another key benefit to AIB is that the service is available 24 hours a day, seven days a week.<br />

214


Karen Neville<br />

AIB plans to further deploy this soluti<strong>on</strong> to another 15,000 staff across its head office, branch and<br />

capital markets staff.<br />

Identity <strong>Management</strong> System for Colombian Banking Industry<br />

A financial instituti<strong>on</strong> in Colombia has rolled out a multi-biometric identity management system. The<br />

soluti<strong>on</strong> will enable the enrolment, management and verificati<strong>on</strong> of pers<strong>on</strong>al identities to eliminate<br />

identity fraud and financial loss. The system will utilize face, fingerprint and signature biometric<br />

identifiers as well as provide real-time duplicity check up<strong>on</strong> enrolment to avoid multiple enrolment<br />

attempts and c<strong>on</strong>firm that account holders are who they claim to be.<br />

Citibank Singapore Rolls Out Biometrics<br />

Citibank Singapore has rolled out Pay By Touch's biometric payment services to its Citibank Clear<br />

Platinum cardholders. Cardholders are invited to enrol in the biometric credit card service, and pay for<br />

goods and services with the touch of a finger. As the credit card is very much a part of daily lives,<br />

c<strong>on</strong>sumers want greater flexibility in making payments. This is the ultimate in c<strong>on</strong>venience – the<br />

ability to make credit card transacti<strong>on</strong>s with just a finger scan. The new biometric payment service is<br />

available to Citibank Singapore's Clear Platinum cardholders as part of the launch of its new Clear<br />

Platinum card. Platinum cardholders will be able to make cardless credit card transacti<strong>on</strong>s at retail<br />

outlets such as music and IT stores, as well as clubs, restaurants and cinemas. Enrolment takes<br />

minutes, and requires <strong>on</strong>ly a government-issued photo ID, a Citibank Clear Platinum credit card and a<br />

secure finger scan. Cardholders also create a seven-digit Pers<strong>on</strong>al Search Number, which facilitates<br />

the use of their scanned finger image to authenticate payment transacti<strong>on</strong>s.<br />

The Bank of Currituck Utilises Fingerprints for Physical Access<br />

The Bank of Currituck recently purchased and implemented US Biometrics' AccessQ system for<br />

c<strong>on</strong>trolling access to various doorways with fingerprint biometric technology. US Biometrics worked<br />

with The Bank of Currituck to specify and implement an access c<strong>on</strong>trol system, including hardware<br />

and software, with the goal of biometrically c<strong>on</strong>trolling strategic doorways in their facility. The<br />

installati<strong>on</strong> of these devices provides an effective biometric perimeter around the bank but it is also<br />

adds a new level of c<strong>on</strong>venience for employees. They no l<strong>on</strong>ger have to carry cards or badges. The<br />

soluti<strong>on</strong> is designed to be a secure, c<strong>on</strong>venient, and cost-effective alternative to passwords, badges,<br />

swipe cards and PINs. It allows for the storing of credentials including fingerprint profiles, names,<br />

addresses, and employee informati<strong>on</strong> as well as providing a mechanism for scheduling access based<br />

<strong>on</strong> authorizati<strong>on</strong> levels and the time of day. Additi<strong>on</strong>ally reporting allows, through a web browser, to<br />

view who accessed which devices at what time as well as provide some basic statistics about the<br />

system's usage.<br />

First Facial Recogniti<strong>on</strong> System for Online Banking<br />

Las Vegas, CA - Cogneto with Cognitec Systems has integrated facial biometrics into UNOMI,<br />

Cogneto's c<strong>on</strong>sensus-based risk adaptive authenticati<strong>on</strong> soluti<strong>on</strong> to bring facial recogniti<strong>on</strong> to the<br />

<strong>on</strong>line banking market. Cogneto will offer facial recogniti<strong>on</strong> as an additi<strong>on</strong>al layer of authenticati<strong>on</strong>,<br />

increasing the security level for UNOMI users. Using their own webcams, users can receive a higher<br />

level of identity assurance for their <strong>on</strong>line bank accounts. Cognitec's software matches images of<br />

users with photos taken by UNOMI during the normal log-in process. This applicati<strong>on</strong> will be<br />

integrated into UNOMI's c<strong>on</strong>sensus model, which uses informati<strong>on</strong> from multiple factors to determine<br />

a user's security and risk rating. As webcams are now standard <strong>on</strong> many PCs, this is an easy way for<br />

banks to manage risk and increase the level of security <strong>on</strong> certain customer accounts. The integrati<strong>on</strong><br />

of facial recogniti<strong>on</strong> allows for potential expansi<strong>on</strong> of the product into ATMs and in-branch verificati<strong>on</strong>.<br />

Using an individual's <strong>on</strong>line photo verificati<strong>on</strong> records, a bank teller or ATM will have an additi<strong>on</strong>al<br />

method to accurately identify people and further avoid identity fraud.<br />

Paycheck Secure Service for Underbanked C<strong>on</strong>sumers<br />

Paycheck Secure is a biometric check-cashing service that can help banks and credit uni<strong>on</strong>s<br />

generate additi<strong>on</strong>al n<strong>on</strong>-interest fee-based income while decreasing the risk of fraud and meeting<br />

regulatory compliance requirements. The service lets people cash checks using a simple finger scan<br />

to authenticate their identity. Already used by more than three milli<strong>on</strong> c<strong>on</strong>sumers in 1,700 retail<br />

locati<strong>on</strong>s nati<strong>on</strong>wide, Paycheck Secure is a popular and widely adopted biometric check cashing<br />

service. The service is aimed at helping banks and credit uni<strong>on</strong>s reach the more than 45 milli<strong>on</strong><br />

'underbanked' c<strong>on</strong>sumers who are presently underserved by financial instituti<strong>on</strong>s. According to the<br />

Center for Financial Services Informati<strong>on</strong> (CFSI), this group represents 40 milli<strong>on</strong> households and<br />

spends $10.9 billi<strong>on</strong> per year <strong>on</strong> 324 milli<strong>on</strong> alternative financial transacti<strong>on</strong>s. A 2006 study by<br />

215


Karen Neville<br />

BearingPoint and Visa revealed that the underbanked generate $1.1 trilli<strong>on</strong> in income. For instance,<br />

the historically underbanked Hispanic community will drive 50 percent of the growth in retail banking<br />

in the decade to come.<br />

Payment Service for Supermarket Retailers<br />

Thriftway Supermarket in Seattle, Washingt<strong>on</strong> was the first retailer to install a new point-of-sale<br />

applicati<strong>on</strong> that provides a turnkey soluti<strong>on</strong> for Pay By Touch(TM), a popular biometric payment<br />

service that lets shoppers make purchases with the touch of a finger. The applicati<strong>on</strong> was<br />

implemented in 13 checkout lanes. It enables Thriftway to handle biometric payments faster,<br />

enhancing customer service. A supermarket can quickly deploy Pay By Touch to support the growing<br />

demand for reduced costs and increased security in electr<strong>on</strong>ic payment transacti<strong>on</strong>s. For simplicity<br />

and added security, Thriftway is using IBM e-business Hosting Services. As part of an IBM Software<br />

as Services soluti<strong>on</strong>, shoppers' Pay By Touch digital wallet informati<strong>on</strong> is securely stored off-site at<br />

IBM data centres. IBM Software as Services offers clients lower costs that are aligned with usage,<br />

minimal upfr<strong>on</strong>t expense, rapid implementati<strong>on</strong> and reduced risk. Shoppers at Thriftway Supermarket<br />

can now purchase groceries by providing a simple fingerprint image that is linked to their financial<br />

accounts and loyalty programs. The shopper selects which account they want to use, the transacti<strong>on</strong><br />

is processed as if a card or check has been presented, and rewards points are automatically<br />

recognized and awarded. The checkout routine is faster, helps protect customers from identity theft<br />

and eliminates the need for shoppers to carry cash, multiple credit cards, or bring their check book.<br />

Kiosk-Based Banking<br />

Real-Time Data <strong>Management</strong> Systems and SAFLINK have a l<strong>on</strong>g-standing relati<strong>on</strong>ship to provide<br />

credit uni<strong>on</strong>s with full-service kiosks that essentially act as self-service branch facilities. Credit Uni<strong>on</strong><br />

customers can open accounts, cash checks, apply for loans and make CD and deposits. The kiosks<br />

enable Credit Uni<strong>on</strong>s to leverage opportunities for new membership at locati<strong>on</strong>s that are lucrative but<br />

which cannot cost-justify the establishment of a new branch. They also provide 24-hour service for<br />

customers who cannot access branches during regular business hours. Customer satisfacti<strong>on</strong> levels<br />

have been extremely high for Credit Uni<strong>on</strong>s that have deployed these kiosks.<br />

Retail Point Of Sale (POS)<br />

In the US more than 500 milli<strong>on</strong> checks are forged each year. Hernd<strong>on</strong>, Virginia based BioPay<br />

addressed this issue with a biometric a finger scan system that allows retail merchants to share<br />

negative customer financial transacti<strong>on</strong> informati<strong>on</strong> in real time, significantly reducing risk of “bad<br />

check” acceptance. The company’s initial focus has been in the c<strong>on</strong>venience store market where<br />

check cashing offers a lucrative business opportunity as l<strong>on</strong>g as fraud can be c<strong>on</strong>tained. BioPay is<br />

beginning to see momentum in this market. More than 4.5 milli<strong>on</strong> check-cashing transacti<strong>on</strong>s worth<br />

more than $2 billi<strong>on</strong> have been completed using their systems.<br />

3.2 Biometrics acceptance<br />

Innovati<strong>on</strong> is defined as ‘…an idea, practice or object perceived as new by the individual’ (Roger<br />

1962). For an organisati<strong>on</strong> or instituti<strong>on</strong>, it is any product, input, process, service or technology that<br />

the organisati<strong>on</strong> perceives as new. In the case of the Financial Services Industry biometrics are<br />

regarded as innovative and not as a form of identity management but as a vehicle in targeting new<br />

markets such as mobile payments/ banking, in house c<strong>on</strong>trol and niche markets such as the<br />

underbanked. Figure 2 illustrates the variables necessary for the successful acceptance of an<br />

innovative technology which helps technology implementers (developers, banks, organizati<strong>on</strong>s)<br />

advance the diffusi<strong>on</strong> of selected technologies. The model is based <strong>on</strong> both the characteristics of<br />

perceived adopti<strong>on</strong> as identified by practiti<strong>on</strong>ers through deployments. The following are the eight<br />

characteristics recognised to enhance the rate and effectiveness of diffusi<strong>on</strong>: (1) relative advantage,<br />

(2) compatibility, (3) trial ability, (4) ease of use, (5) visibility, (6) result dem<strong>on</strong>stratability, (7) image<br />

and (8) voluntariness.<br />

The first characteristic is the relative advantage of the innovati<strong>on</strong> over the idea it replaces (in for<br />

example Financial Services) including ec<strong>on</strong>omic profitability, c<strong>on</strong>venience and/or other benefits. A<br />

technology is more likely to be accepted if it is perceived as bringing advantages. The sec<strong>on</strong>d<br />

characteristic is the compatibility of biometrics (innovati<strong>on</strong>) with the existing values, past experiences<br />

and needs of adopters.<br />

216


Karen Neville<br />

Figure 2: Modified model of intenti<strong>on</strong> to adopt using PCI measures<br />

People (organizati<strong>on</strong>s/ customers/ banks/ merchants) are more likely to adopt technology if it is<br />

functi<strong>on</strong>ally compatible to those previously adopted and is c<strong>on</strong>sistent with the existing values, needs<br />

and past experiences of adopters. Therefore ease of transiti<strong>on</strong> from <strong>on</strong>e service /product (for example<br />

the integrati<strong>on</strong> of biometrics with smart-cards and mobile ph<strong>on</strong>es in banking) to another is vital in<br />

acceptance. The third characteristic relates to the level of complexity or the ease with which an<br />

innovati<strong>on</strong> can be understood. As biometrics have been prevalent in government identity systems for<br />

years it is relativity comm<strong>on</strong> to milli<strong>on</strong>s of people. The fourth and fifth related characteristics are<br />

described as trialability, or the degree to which adopters can implement an innovati<strong>on</strong>, <strong>on</strong> an<br />

experimental basis and observability, or the extent to which results of an innovati<strong>on</strong> are visible to<br />

others. Secti<strong>on</strong> 3.1 described deployments in the Financial services which have proven advantageous<br />

in performance, flexibility, scalability and in counteracting the challenges. These factors are not just<br />

dependent <strong>on</strong> the specific nature of the innovati<strong>on</strong> but also <strong>on</strong> the specific characteristics of the<br />

adopting group. Ease-of-use is vital for end-user acceptance. But in the case of biometrics customer<br />

acceptance is primarily high. Visibility is the extent to which the innovati<strong>on</strong> is perceived to have<br />

diffused. Result dem<strong>on</strong>strability is how well the benefits and results are recognised by the potential<br />

adopter (financial instituti<strong>on</strong>). Image is the status or prestige that the adopter thinks they will gain from<br />

the innovati<strong>on</strong>. Voluntariness is the degree to which an individual has the choice whether or not to<br />

adopt the innovati<strong>on</strong> which ultimately depends <strong>on</strong> the competitiveness of the adopter. The value of<br />

innovative applicati<strong>on</strong>s is dependent up<strong>on</strong> its adopti<strong>on</strong> and acceptance by relevant parties (users,<br />

organizati<strong>on</strong>s). Critical mass, the point at which”....enough individuals have adopted an interactive<br />

innovati<strong>on</strong> to cause the perceived cost-benefit of adopti<strong>on</strong> to change from negative to positive so that<br />

the innovati<strong>on</strong>’s rate of adopti<strong>on</strong> becomes self-sustaining” is therefore vital so that a new technology<br />

is ec<strong>on</strong>omically viable (Roger 1991). Therefore <strong>on</strong>e could not but argue that biometrics is accepted<br />

given the level of critical mass currently achieved.<br />

4. C<strong>on</strong>clusi<strong>on</strong>s<br />

The adopti<strong>on</strong> of biometrics within financial services to support existing and target new markets is<br />

inevitable as the volume of successful deployments increases. This will result in industry comfort and<br />

c<strong>on</strong>sumer acceptance. Additi<strong>on</strong>ally the number of applicati<strong>on</strong>s (and their uses) will expand as<br />

biometrics will become a ubiquitous comp<strong>on</strong>ent of the financial services infrastructure. The applicati<strong>on</strong><br />

of biometrics ranges from authenticating inter-bank transfers and accessing local savings bank<br />

accounts to the weekly purchase of groceries at the supermarket, pers<strong>on</strong>al identificati<strong>on</strong> and<br />

transacti<strong>on</strong> processing. Figure 2 illustrates the characteristics necessary for adopti<strong>on</strong> as identified in<br />

research while emphasing the importance of a Trusted Third Party for authenticati<strong>on</strong> and assurance<br />

and a collaborative body to interface with the banks and c<strong>on</strong>tent providers to provide guidance for<br />

both deployment and standardisati<strong>on</strong>. Ultimately biometrics will become as familiar and trusted as<br />

displaying a driver’s license or making a purchase with an ATM card. Additi<strong>on</strong>ally figure 2 illustrates<br />

217


Karen Neville<br />

the numerous types of biometric deployments categorised according to the type of biometric used. All<br />

of these implementati<strong>on</strong>s highlighted the numerous advantages associated with the adopti<strong>on</strong> of the<br />

technology for the industry. Financial services like any of the other industries availing of the<br />

technology is driven and challenged by key issues such as interoperability and costs. However it is<br />

TTPs (Trusted Thirst Parties) which can provide the technology, expertise and interoperability<br />

necessary to successfully meet the requirements of any potential customer. As illustrated in Figure 3<br />

a TTP will not <strong>on</strong>ly provide the ultimate approach to identity management but the characteristics<br />

necessary for acceptance of biometric to any and all stakeholders.<br />

Figure 3: Providing expertise in identity management<br />

References<br />

WWW1: “10 Emerging Technologies That Will Change the World”, MIT Technology Review (January – February,<br />

2001 – www.techreview.com).<br />

WWW2: 2007 Global Security Survey: “The Shifting Security Paradigm”. Deloitte, 2007<br />

www.deloitte.com/assets/Dcom-Shared% 20Assets/Documents/dtt_gfsi_GlobalSecuritySurvey_<br />

20070901.pdf).<br />

2007 Payments System Research Briefing: “Complex Landscapes in Japan, South Korea and the United States”,<br />

Federal Reserve Bank of Kansas City.<br />

Rogers, E. (1962) “Diffusi<strong>on</strong> of Innovati<strong>on</strong>s”, New York: The Free Press.<br />

Rogers, E. (1991) “The Critical mass in the diffusi<strong>on</strong> of interactive technologies in organisati<strong>on</strong>s”, Harvard<br />

Business Research Colloquium, Publishing Divisi<strong>on</strong>, Harvard Business School, Bost<strong>on</strong>.<br />

218


Did you get Your Facebook Sessi<strong>on</strong> Completed?<br />

Markku Nurminen<br />

University of Turku, Turku, Finland<br />

markku.nurminen@utu.fi<br />

Abstract: Traditi<strong>on</strong>ally, most informati<strong>on</strong> systems were used at work. These systems offered means and tools for<br />

people that enabled them to perform their work more effectively, easily, or with higher quality outcome. One of<br />

the most important criteria of evaluati<strong>on</strong> of such systems is derived from the c<strong>on</strong>tributi<strong>on</strong> of the system to the<br />

work objectives of its users. Such evaluati<strong>on</strong> can be started by embedding the IS acti<strong>on</strong>s and operati<strong>on</strong>s as<br />

inherent parts in the users’ work processes. This means that the ultimate criteria of the system come from the<br />

outside of the system itself. It is interesting that the absence of quality is easier to observe than its presence.<br />

Today, many users spend their time with surfing in the Internet, playing games, or attending at social media.<br />

Electr<strong>on</strong>ic services give <strong>on</strong>e more use situati<strong>on</strong> where this kind of traditi<strong>on</strong>al goal-oriented evaluati<strong>on</strong> no l<strong>on</strong>ger<br />

seems to be sufficient. In all of these (and many other) use situati<strong>on</strong>s the objectives are not necessarily clearly or<br />

explicitly defined and therefore it is difficult to evaluate, to what extent the objectives have been fulfilled. Such<br />

activities are said to be weakly purposeful. The added value created cannot be observed in the external object of<br />

the work, the change is often likely to take place inside the actor him/herself, for example as a use experience or<br />

improvement of the competence through learning. This paper addresses the problem characterised above that<br />

goes deep to the core problems of evaluati<strong>on</strong>. First the generic c<strong>on</strong>cept of purposeful activity will be discussed.<br />

Work lends itself to be analysed in terms of three modalities of work: individual work, collective work and<br />

services. Then the electr<strong>on</strong>ic services and IT-services are analysed in terms of this generic c<strong>on</strong>cept, paying<br />

special attenti<strong>on</strong> to self-services. Finally, the main problem of evaluati<strong>on</strong> of IT-artefacts used for weakly<br />

purposeful activities is discussed and some guidelines for evaluati<strong>on</strong> are derived. The c<strong>on</strong>tributi<strong>on</strong> of the paper is<br />

in its c<strong>on</strong>ceptual analysis, and it is <strong>on</strong>ly indirectly based <strong>on</strong> own empirical work and material.<br />

Keywords: self-service, evaluati<strong>on</strong> criteria, purposeful activity, added value, use experience<br />

1. Introducti<strong>on</strong><br />

Evaluati<strong>on</strong> of an Informati<strong>on</strong> System is about success and failure. Because the introducti<strong>on</strong> of an IS<br />

always carries a cost, it should also create some benefit in order to be justified. The comparis<strong>on</strong> of<br />

costs and benefits happens typically both before and after the introducti<strong>on</strong>. In the ex ante evaluati<strong>on</strong><br />

the motivati<strong>on</strong> is in the desire to make good and well-informed decisi<strong>on</strong>s about the future investment,<br />

and in the ex post evaluati<strong>on</strong> the rati<strong>on</strong>ale comes from the need for follow-up: did the expectati<strong>on</strong>s<br />

come true? In the best case this will lead to better decisi<strong>on</strong>s in the future investment rounds.<br />

In both ex-ante and ex-post evaluati<strong>on</strong> it is crucial to base the evaluati<strong>on</strong> <strong>on</strong> a thorough understanding<br />

about the mechanism of the creati<strong>on</strong> of the benefits by means of the informati<strong>on</strong> technology.<br />

Otherwise the informati<strong>on</strong> system remains as a black box that is expected to produce added value<br />

due to a mythical belief. There is no generally accepted understanding of such a mechanism. One<br />

frequently occurring c<strong>on</strong>cern is that the framework of expectati<strong>on</strong>s and follow-up is too narrow, in<br />

particular too technically determined. It is justified to call such failures “expectati<strong>on</strong> failures” (Lyytinen<br />

1988).<br />

Jan.L. Andresen (2001) has selected four evaluati<strong>on</strong> methods that are representative am<strong>on</strong>g a large<br />

set of approaches – or perhaps serve as an abstracti<strong>on</strong> or aggregati<strong>on</strong> of the otherwise l<strong>on</strong>g list of<br />

them. The four methods are:<br />

Net Present Value (NPV)<br />

Informati<strong>on</strong> Ec<strong>on</strong>omics (IE)<br />

Critical Success Factors (CSF)<br />

Measuring the Benefits of IT Innovati<strong>on</strong> (MBITI)<br />

The first <strong>on</strong>e (NPV) is obviously derived from general investment evaluati<strong>on</strong>s. The m<strong>on</strong>etary interest<br />

is open, but the underlying mechanism is entirely bracketed. The sec<strong>on</strong>d <strong>on</strong>e (IE) allows a broader<br />

spectrum of factors to be c<strong>on</strong>sidered <strong>on</strong> the side of traditi<strong>on</strong>al cost-benefit analysis, such as strategic<br />

match and competitive advantage. The model needs, however, all factors to be quantified and does<br />

not directly address the problem of articulating the value-creating mechanism. The measuring needed<br />

for quantificati<strong>on</strong> is extremely difficult and must often be based <strong>on</strong> people’s opini<strong>on</strong>s than real<br />

changes in the business processes. The Critical Success Factor (CSF) approach allows deeper<br />

participati<strong>on</strong> of the key pers<strong>on</strong>s in the evaluati<strong>on</strong> process. A detailed model of the effecting<br />

219


Markku Nurminen<br />

mechanism is, however, difficult to combine with the critical success factors, because such factors<br />

necessarily are defined at a high level of abstracti<strong>on</strong> and large business units. Like in the previous<br />

approach, the reality as the object of evaluati<strong>on</strong>, is often c<strong>on</strong>veniently replaced by the opini<strong>on</strong>s about<br />

the reality. The fourth approach MBITI comes perhaps closest to our idea of addressing the mediating<br />

mechanism. This approach analyses all the main business processes and identifies benefits in terms<br />

of efficiency, effectiveness, and performance. The straightforward approach also aims at finding<br />

means of measuring the benefits and a pers<strong>on</strong> resp<strong>on</strong>sible for the achievement of these benefits. Yet<br />

it seems that the investment is regarded as something that is separate from the basic activities of the<br />

company. This is exactly the problem in the vast majority of all evaluati<strong>on</strong> approaches known by the<br />

author.<br />

2. Approach to evaluati<strong>on</strong><br />

2.1 Appropriati<strong>on</strong><br />

When informati<strong>on</strong> technology artefacts and informati<strong>on</strong> systems are implemented, they should be<br />

embedded in the activities of the organisati<strong>on</strong> that uses them. This means that each user should<br />

appropriate (Baillette and Kimble) the artefact so that it becomes her property in terms of c<strong>on</strong>trol and<br />

mastery so that it is practically impossible to observe or analyse it separately from the rest of her<br />

work. We decide to call this characteristic as the “Inseparability principle”: IT use and other work<br />

tasks c<strong>on</strong>stitute an inherent whole. For the evaluati<strong>on</strong> interest this means that whenever the evaluator<br />

is able to observe and analyse the IS as a separate entity, this proves that the implementati<strong>on</strong> has<br />

failed and the evaluati<strong>on</strong> therefore will necessarily give a poor degree. This statement does no imply<br />

that evaluati<strong>on</strong> of IT would be a missi<strong>on</strong> impossible by its nature. But it means that an evaluati<strong>on</strong><br />

attempt makes sense <strong>on</strong>ly when it is integrated with the evaluati<strong>on</strong> of all the activities that the system<br />

is supposed to support.<br />

2.2 Work-centred evaluati<strong>on</strong><br />

The criteria of evaluati<strong>on</strong> now come from the criteria of the work activities, that (paradoxically enough)<br />

reside outside the informati<strong>on</strong> system as a technical c<strong>on</strong>struct. In order to extract such criteria, we<br />

have to describe the c<strong>on</strong>tents of the work activities in terms that are independent of the technical<br />

soluti<strong>on</strong> used for its performance. Often it is useful to refer to the abstract functi<strong>on</strong>s of IT such as data<br />

storage, communicati<strong>on</strong>, and processing informati<strong>on</strong> instead of actual c<strong>on</strong>crete means and tools.<br />

From this perspective, the previous situati<strong>on</strong> (with the old IS or without any) and the future (current)<br />

situati<strong>on</strong> appear as two alternative ways to solve the same problem of job design. In other words, they<br />

are directly comparable. In an ex ante evaluati<strong>on</strong>, alternative future system may be compared<br />

pairwise in order to find the most promising candidate. This kind of approach is extremely useful,<br />

since the origins of the expected added value are made visible.<br />

2.3 Relati<strong>on</strong>ship between IS and work activities<br />

The argumentati<strong>on</strong> above indicates that in this work we see the c<strong>on</strong>necti<strong>on</strong> between work and<br />

informati<strong>on</strong> technology according to the inseparability principle. One or another c<strong>on</strong>ceptualisati<strong>on</strong> of<br />

this relati<strong>on</strong>ship is necessary, if we want to perform any evaluati<strong>on</strong> that pays attenti<strong>on</strong> to the<br />

objectives of the work and business activities.<br />

2.4 Work role<br />

The basic building block of this c<strong>on</strong>ceptualisati<strong>on</strong> of the relati<strong>on</strong>ship between IT and work is the “Work<br />

Role”. In my early c<strong>on</strong>ceptualisati<strong>on</strong>s I was striving after a richer noti<strong>on</strong> of work than process thinking<br />

al<strong>on</strong>e can give and therefore I started in oppositi<strong>on</strong> to business processes. Recently, I have tried to<br />

find a new understanding of work that is not a mere antithesis of business processes, but is<br />

something more generic, giving some kind of synthesis. The new formulati<strong>on</strong> acknowledges the huge<br />

power of business processes and their representati<strong>on</strong>s, but does not accept the claim that the<br />

process view is the whole truth<br />

2.4.1 C<strong>on</strong>structi<strong>on</strong> of the work role<br />

The skelet<strong>on</strong> of the work role c<strong>on</strong>sists of <strong>on</strong>e or more business processes or parts of them assigned<br />

to the work role. As a part of this, related IS tasks are regarded as an inherent part of the process.<br />

220


Markku Nurminen<br />

This offers the key to the mechanism of the creati<strong>on</strong> of the benefit of IT, which is crucial for the<br />

evaluati<strong>on</strong> interest.<br />

The work role is not <strong>on</strong>ly processes, but the employees have also other duties that can be collected<br />

under the general noti<strong>on</strong> of resp<strong>on</strong>sibility. Some more detailed issues <strong>on</strong> such resp<strong>on</strong>sibility can be<br />

identified:<br />

The successful performance al<strong>on</strong>e is not always enough. The actor should take care also of the<br />

site of the work situati<strong>on</strong> after processes. The IT tasks are normally expected to leave the data<br />

bases in the correct state, but the resp<strong>on</strong>sible actor is supposed keep his eyes open and detect<br />

any errors or other deviati<strong>on</strong>s that may require attenti<strong>on</strong> and perhaps corrected.<br />

The resp<strong>on</strong>sibility of the actor does not c<strong>on</strong>cern the primary object of the work <strong>on</strong>ly, also the work<br />

envir<strong>on</strong>ment and the tools have to be observed, and any acti<strong>on</strong> of maintenance or cleaning<br />

should be taken if it is likely to promote the future performance. The knowledge and skill of the<br />

actor himself is naturally <strong>on</strong>e object of maintenance, occasi<strong>on</strong>ally requiring learning or educati<strong>on</strong>.<br />

The actor is supposed to maintain a sufficient level of awareness in his work envir<strong>on</strong>ment. Any<br />

factor that may effect to an excepti<strong>on</strong>al routing or performance of the work should be<br />

acknoweledged and necessary rearticulati<strong>on</strong> d<strong>on</strong>e. One important object of awareness is the<br />

recogniti<strong>on</strong> of errors and excepti<strong>on</strong>s.<br />

The stereotypical noti<strong>on</strong> of business processes typically assume that a process is triggered by an<br />

event coming from outside, such as the customer entering and requiring attenti<strong>on</strong>. The<br />

resp<strong>on</strong>sible actor may identify the need for example for a maintaining acti<strong>on</strong> and initiate an<br />

appropriate process, or improvise a new and relevant <strong>on</strong>e.<br />

All these points assume that the actor is trusted and familiar with the best practices in and around<br />

his work role; then he can take the resp<strong>on</strong>sibility given to him.<br />

2.4.2 The three modalities of work<br />

The mechanism of creati<strong>on</strong> and collecti<strong>on</strong> of the added value enabled by informati<strong>on</strong> systems varies<br />

according to the actual modality of work. People work in <strong>on</strong>e or more of the three modalities. Or<br />

probably all three are present in most work situati<strong>on</strong>s, even if <strong>on</strong>e of them may have a dominant role.<br />

The set of modalities lends itself to be used in the analysis and understanding of the structure and<br />

functi<strong>on</strong>ing of even complicated settings in working life and, hopefully, also private spheres of worklike<br />

purposeful activities. The three modalities are<br />

Individual work<br />

Collective work<br />

Services<br />

In individual work modality the actor works <strong>on</strong> an individually assigned work process. Some tasks are<br />

performed by means of IT. Before starting the acti<strong>on</strong> the actor articulates the current task and tailors it<br />

to fit to the situated factors and his own competence. Requirements (specificati<strong>on</strong>s) for the task are<br />

often mediated by means of the IT as well as the reporting of the completed task. The informati<strong>on</strong><br />

system is seen as a “tool” for the actor. In collective work modality the actors share their work tasks<br />

jointly. Thus they also have to articulate the work jointly. Not <strong>on</strong>ly each individual articulates his lot but<br />

also the collaborati<strong>on</strong> has to be articulated. This collective aspect is the most frequently used noti<strong>on</strong><br />

of articulati<strong>on</strong> work (Schmidt and Bann<strong>on</strong>, 1992) according to the introducti<strong>on</strong> of the term by Anselm<br />

Strauß. The informati<strong>on</strong> system typically mediates the collaborati<strong>on</strong> within the collective and it can be<br />

understood by the metaphor “Medium”.<br />

Individual and collective work can be seen as two poles that span a field of dialectical tensi<strong>on</strong><br />

between them. This implies that n<strong>on</strong>e of these two can be thoroughly understood without a reference<br />

to the other. The third modality, service, is not equally obvious, somebody has suggested that service<br />

could be seen as a specific form of collaborative work in which both the producer and the customer<br />

jointly create the added value that is the ultimate purpose of the service. This suggesti<strong>on</strong> ignores,<br />

however, the fundamental gap between the two parties of the service. In collective work the team (or<br />

group, if you prefer) is characterised by shared objectives to be strived after. In services the two<br />

parties are not sitting <strong>on</strong> the same side of the counter, but they are <strong>on</strong> the opposite sides of it. This<br />

asymmetry is further emphasised by the fact that the added value created typically will be the benefit<br />

221


Markku Nurminen<br />

of the customer or his processes. The two parties may even have c<strong>on</strong>flicting interests: the producer<br />

may want to sell the service at a high price whereas the customer often is willing to pay a lower price.<br />

The character of services is thus distinct from the character of collective work. The distinctive feature<br />

is probably the exchange. The difference is more clearly visible in B2B services than in B2C services.<br />

In the B2B services the customer has externalised some parts of its own processes to be ordered<br />

from the provider. In other words they have decided to select the alternative BUY instead of MAKE<br />

(the two main opti<strong>on</strong>s of exchange presented in the Transacti<strong>on</strong> Cost Theory (Williams<strong>on</strong> 1981))<br />

these parts themselves. In the services two otherwise distinct processes (or practices) of the two<br />

parties meet each other to enable the delivery of the service to happen. Successful delivery integrates<br />

the service as a part of the customer’s processes. After the delivery the two processes of the both<br />

parties c<strong>on</strong>tinue their own life cycles. And c<strong>on</strong>sequently, the customer is the side that is supposed to<br />

be the beneficiary of the added value created in the service (the provider receives the payment). The<br />

articulati<strong>on</strong> in services takes a form of c<strong>on</strong>tract due to the tensi<strong>on</strong> between the parties.<br />

2.4.3 Evaluati<strong>on</strong><br />

Each modality of work is based <strong>on</strong> the noti<strong>on</strong> of purposeful activity of the employees working in the<br />

organisati<strong>on</strong> or with their service customers. When the IS functi<strong>on</strong>s are embedded in the processes<br />

and activities of the users, we will be able to evaluate the degree to which the IT artefact c<strong>on</strong>tributes<br />

to the objectives of the activity under study. Important is that the success or failure is evaluated in<br />

terms of the activity’s goals, not in IT specific terms of usability. The “measurement” clearly boils down<br />

to pairwise comparis<strong>on</strong> between the old (old system or manual practice) and new or future practice.<br />

2.5 IT as service and self-service<br />

In the era of electr<strong>on</strong>ic services we have to make it clear for ourselves, how we understand the role of<br />

IT in such services. It is important to notice that the use of computers is a service in itself. Many of us<br />

still remember those days in the 1950’s and 1960’, when particular organisati<strong>on</strong>s collected around<br />

Computer Centres performed computing service for their customers that delivered their input data<br />

coded in punched cards (later magnetic tapes) and received sheets printed by high speed printers.<br />

Many specialised supporting subservices flourished as parts in fluent service chains, such as keypunching,<br />

transportati<strong>on</strong>, etc.<br />

Electr<strong>on</strong>ic data transmissi<strong>on</strong> and <strong>on</strong>-line real time systems operated by time-sharing operating<br />

systems turned this whole service chain into a sequence of self-services. One bank clerk could<br />

perform the coding of the transacti<strong>on</strong> and its transformati<strong>on</strong> to machine-readable form, transfer these<br />

data to be processed without practically any human interventi<strong>on</strong> of the computer operator. The bank<br />

clerk produced the IT-services in a self-service mode. The IT artefact had become a tool for her that<br />

was in fairly good c<strong>on</strong>trol of him.<br />

The next step in the development of e-services happened when the bank’s customer started to work<br />

as the operator in entering the details of the desired transacti<strong>on</strong>, first in the corner of the bank hall or<br />

vestibule, later at home with the broadband networking. Here the entire banking service was moved to<br />

self-service. At the first glimpse this seems to be in c<strong>on</strong>tradicti<strong>on</strong> with our preliminary analysis of the<br />

service as <strong>on</strong>e of the three modalities of work. The emerging situati<strong>on</strong> has sometimes been<br />

misinterpreted by saying that the customer take over the earlier work role kept by the bank clerk. This<br />

is necessarily wr<strong>on</strong>g, because the bank does not afford to give its customers the privileges of its<br />

employees (as the part of the work role). Rather, it is crucial to maintain a clear distincti<strong>on</strong> between<br />

the two parties: the service provider and the customer. What the customer does when entering the<br />

details of the desired transacti<strong>on</strong> to his screen while doing his home banking, is, that he writes and<br />

delivers the service request to the bank. The bank (or the software representing it) then analyses the<br />

request, gives remarks if needed (e.g. too low balance <strong>on</strong> the account for paying the bill requested),<br />

and finally executes the transacti<strong>on</strong> and gives feedback about it (e.g. a receipt). The fact that there is<br />

no indentifiable pers<strong>on</strong> of flesh and blood <strong>on</strong> the other side of the desk, does not mean that the work<br />

role is entirely missing. Whenever an error or unclear transacti<strong>on</strong> occurs, an actor with sufficient<br />

expertise and privileges will show up and clear the mess. But most of the working operati<strong>on</strong> hours the<br />

resp<strong>on</strong>sible clerk has left the desk for automated processing of routine tasks. This is not much more<br />

complicated than the operati<strong>on</strong> of automatic washing machine, but the point is extremely important for<br />

the phenomen<strong>on</strong> of electr<strong>on</strong>ic services. The work role in such services can be left to be performed by<br />

222


Markku Nurminen<br />

computer software, even if we know quite well, that if something unexpected occurs, there is a rolekeeper<br />

in pers<strong>on</strong> who enters and solves the emerged problems.<br />

Electr<strong>on</strong>ic services in some cases lead to the delivery of physical objects (book store), but the real<br />

power of e-services is visible whenever also the delivery can be electr<strong>on</strong>ic. This happens for example<br />

when <strong>on</strong>e downloads software or music directly from the Internet site.<br />

3. Odyssey of the individual<br />

The decisi<strong>on</strong> to anchor the evaluati<strong>on</strong> to the objectives of work or purposeful is an obvious strength by<br />

addressing the core issues of the introducti<strong>on</strong> of IT and its legitimati<strong>on</strong>: why should we make these<br />

investments. This decisi<strong>on</strong> turns to be the source of problems as so<strong>on</strong> as we start analysing<br />

applicati<strong>on</strong>s that are not directly c<strong>on</strong>nected to such activities. For example, listening to music or<br />

playing games most often are d<strong>on</strong>e during leisure rather than working hours and lack clearly<br />

articulated objectives that could be used as reference for evaluati<strong>on</strong>.<br />

The c<strong>on</strong>ceptualisati<strong>on</strong> of the three modalities of work presented above helps us to make sense of<br />

these activities. We now can regard them as services, more accurately self-services, produced by the<br />

customer himself. In these examples, the customer is also the beneficiary of the added value created<br />

through the producti<strong>on</strong> of the service. He obviously is excited or stimulated through his experience to<br />

the extent that he is willing to pay a relevant price for it. We must be aware that the object of activity<br />

not always is outside the actor. For example, learning, training, entertainment and many other leisure<br />

activities have the aim at changing the state of the actor himself.<br />

It seems to be irrelevant that these use situati<strong>on</strong>s happen outside work organisati<strong>on</strong>s. We just enter<br />

the area of “Everyday Informatics” (EI) (Stringer et al). This does not mean that we must give up the<br />

framework that is based <strong>on</strong> purposeful activity. Our customer wants to get his stimulati<strong>on</strong> and acts<br />

purposefully in order to get it. The main difference is in the observati<strong>on</strong> of the goals and goal<br />

achievement. This time the criteria of success are internal to the customer; it is, indeed, justified to<br />

talk about use experience, since bad music may indicate a failure of this use.<br />

One of the important questi<strong>on</strong>s of Everyday Informatics is in the modelling of human life. In a work<br />

envir<strong>on</strong>ment, it seems to be appropriate to accept social practices to be described in terms<br />

externalised and purpose-oriented representati<strong>on</strong>s, such as process models. The situati<strong>on</strong> is not<br />

necessarily equally straightforward in private areas of life. Of course, there are some areas of<br />

activities that lend themselves to be c<strong>on</strong>ceptualised as projects or similar purposeful activities. Many<br />

pathways of care in the citizen’s health care are good examples of first identifying the current state<br />

(diagnosis) and then designing and deciding a stepwise acti<strong>on</strong> plan that aims at reaching the desired<br />

state. In a recent paper (Lahtiranta and Nurminen 2012)) I have presented a metaphor of “Health<br />

Navigator” that in the spirit of GPS navigator supports the patient-citizen to proceed through the route<br />

specified in the care plan.<br />

Even if many activities can be regarded as purposeful activities that can be supported by means of EI,<br />

we should remember that human life probably cannot be reduced to a set of processes.<br />

3.1 Surfing in the internet<br />

There are two more uses of Informati<strong>on</strong> technology that appear to be still less goal-oriented than<br />

many applicati<strong>on</strong>s of Everyday Informatics. We will discuss two of them: Internet surfing and the<br />

social media (Facebook).<br />

The websites are designed by their publishers. The structure and c<strong>on</strong>tents of the site determines to a<br />

great extent what kind of needs it can meet, the service portfolio. This portfolio is not always visible to<br />

the user, who often starts the surfing without clearly articulated set of objectives. There is then no<br />

direct way to compare the objectives and their fulfilment. Sometimes it is not easy to decide whether<br />

and when the surfing sessi<strong>on</strong> is completed. It is but natural, that most approaches to website<br />

evaluati<strong>on</strong> are derived from rather technical usability characteristics, such as graphics, stability,<br />

compatibility, speed, reliability, accuracy. I am not saying that these factors are irrelevant, but the<br />

mechanism of their impact can be observed as reduced quality of the goal attainment. Yet, I do not<br />

believe that the ending c<strong>on</strong>diti<strong>on</strong> of surfing is entirely subjective, as it was in our discussi<strong>on</strong> about<br />

gaming and other excitement as motivati<strong>on</strong>.<br />

223


Markku Nurminen<br />

The website of an organisati<strong>on</strong> is an intelligent visit card telling its visitors essential informati<strong>on</strong> of<br />

itself. As an example, university websites give answers to many relevant questi<strong>on</strong>s presented by<br />

different stakeholders, the students and teachers in the first place. The website can thus be seen as a<br />

communicati<strong>on</strong> platform between primary stakeholders. If a user has a specific questi<strong>on</strong> in mind, he<br />

can tell whether his surfing sessi<strong>on</strong> was successful in respect to this specific need. Another outcome<br />

may be the answer that this website does not give the answer to this particular questi<strong>on</strong>.<br />

The degree of purposefulness can be increased by adding interactive characteristics to the website.<br />

For example, the student could take more resp<strong>on</strong>sibility of the administrati<strong>on</strong> and performance of his<br />

studies. He could register and download his pers<strong>on</strong>al, tailored study plan and the schedules of the<br />

courses of this plan. Such use could support his navigati<strong>on</strong> through the studying space, provided that<br />

he has his study navigator. The c<strong>on</strong>necti<strong>on</strong> between his own navigator and the website would add the<br />

purposefulness, because now he could really perform both planning and performing his studying<br />

activities (e.g. deliver the exercises electr<strong>on</strong>ically). Then also the evaluati<strong>on</strong> could be based <strong>on</strong> the<br />

evaluati<strong>on</strong> of the success of the goals of the activity.<br />

Similar directi<strong>on</strong> of development could be suggested also for many websites serving e-Government. It<br />

seems that too often the interface is based <strong>on</strong> the assumpti<strong>on</strong> that the customer has <strong>on</strong>ly a dull<br />

terminal without practically any storing or processing capacity. With this equipment, he is supposed<br />

to register to the service provider’s website. But unfortunately, he sometimes is not even given the<br />

opportunity to store his intermediate results for c<strong>on</strong>tinuing his work later.<br />

3.2 Social media<br />

The use of social media can be structured in terms of input and output. Each visit in the Facebook<br />

c<strong>on</strong>nects the user to the community: he can see messages delivered by his friends that tell about the<br />

state and significant events of them. He is also expected to leave some messages about himself in<br />

order to express his current situati<strong>on</strong> and experiences to the community of his friends. On the top of<br />

these two reciprocal modes of acti<strong>on</strong> there is the opportunity to give and receive feedback and thus<br />

maintain the active structure within the community.<br />

Much of these regular visits have the same functi<strong>on</strong> as repeated ph<strong>on</strong>e discussi<strong>on</strong>s with the “How are<br />

you doing?” type issues in the agenda. The maintenance of the awareness of the state of the<br />

community and its individual members also performs the hand-shaking functi<strong>on</strong>: the c<strong>on</strong>necti<strong>on</strong>s are<br />

reliably functi<strong>on</strong>ing.<br />

3.3 Weakly purposeful activities<br />

The principle of anchoring the evaluati<strong>on</strong> of an IS tightly with the purposeful activity that the IT artefact<br />

is supposed to support proved useful, because it enabled the evaluator to identify the point and the<br />

mechanism of the creati<strong>on</strong> of the added value. This approach cannot be directly applied to the<br />

activities that are <strong>on</strong>ly weakly purposeful. Above we learned this to be problematic in two weakly<br />

purposeful use situati<strong>on</strong>s: Internet surfing and social media (Facebook). The term weakly purposeful<br />

describes the situati<strong>on</strong> by indicating that the purpose is not entirely absent. This can be seen in the<br />

fact that the user must decide to initiate the activity. The trigger comes often from the actor himself,<br />

and the objectives for a sessi<strong>on</strong> are perhaps not articulated explicitly. Therefore, it is not easy to<br />

determine, when the activity is completed and can be finished. It may be still more difficult to give an<br />

evaluati<strong>on</strong> of the degree of success (number of like-thumbs in the Facebook).<br />

Yet, there obviously is a reas<strong>on</strong> to initiate a sessi<strong>on</strong>, since otherwise it would never be d<strong>on</strong>e. And<br />

so<strong>on</strong>er or later most sessi<strong>on</strong>s will end, even if this may happen because the actor gets tired rather<br />

than feels the goals fulfilled. The incentives may be in the experienced social pressure rather than the<br />

actor’s true need to get something accomplished. These observati<strong>on</strong>s lead us to psycho-social<br />

analyses of IT uses and their motivati<strong>on</strong>s. Nevertheless, something is d<strong>on</strong>e during every sessi<strong>on</strong>; it<br />

would obviously be misleading to regard such acti<strong>on</strong> as meaningless. On the other hand, traditi<strong>on</strong>al<br />

usability characteristics (speed, c<strong>on</strong>sistency, easy navigati<strong>on</strong>, etc.) al<strong>on</strong>e are not sufficient to tell<br />

anything essential of these weakly purposeful acti<strong>on</strong>s. This leaves us with a challenge to study this<br />

use of IT in more depth. One hint for such studies might be to select an entire sessi<strong>on</strong> to the basic<br />

unit of analysis.<br />

224


4. C<strong>on</strong>clusi<strong>on</strong>s<br />

Markku Nurminen<br />

The evaluati<strong>on</strong> of informati<strong>on</strong> cannot ignore the objectives of the activity it is intended to support. The<br />

usability characteristics of the system are not sufficient al<strong>on</strong>e if the user cannot perform his work with<br />

acceptable quality and efficiency. This statement calls for two fr<strong>on</strong>tiers of future research. The first is<br />

the generati<strong>on</strong> of the representati<strong>on</strong> of the objectives of work in a form that supports the evaluati<strong>on</strong><br />

process better than today. The sec<strong>on</strong>d is the c<strong>on</strong>ceptualisati<strong>on</strong> of the evaluati<strong>on</strong> criteria for situati<strong>on</strong>s<br />

of weakly purposeful activities. In the subsequent discussi<strong>on</strong> some preliminary ideas for both are<br />

suggested.<br />

5. Discussi<strong>on</strong><br />

During the requirement specificati<strong>on</strong> the objectives for the system are specified, often in detail. These<br />

objectives are not always identical, not even c<strong>on</strong>sistent with the objectives of the activity to be<br />

supported. At the best the system objectives are derived from the activity objectives with too little<br />

c<strong>on</strong>siderati<strong>on</strong> of the mechanism of transforming the system benefits back to the activity benefits. In<br />

this paper I have suggested the noti<strong>on</strong> of work role that is richer than process model to be used as a<br />

point of departure for creating the representati<strong>on</strong> of human work that, together with the three<br />

modalities of work lends itself for an operati<strong>on</strong>al means also for evaluati<strong>on</strong> purposes.<br />

The weakly purposeful activities is a serious challenge that is too many-sided to get the soluti<strong>on</strong> here.<br />

Since I in this paper have been in favour of a representati<strong>on</strong> of objectives in str<strong>on</strong>gly purposeful<br />

activities, I would be unhappy of getting into fully psychologised approach to evaluati<strong>on</strong> of weakly<br />

purposeful activities. Quite c<strong>on</strong>trary, I believe that there is an implicit structure of objectives. Perhaps<br />

we could use some suggested approaches in Knowledge <strong>Management</strong> schools to making the actors’<br />

tacit knowledge visible for others. The most promising strategy for future research <strong>on</strong> this problem is<br />

in my thinking based <strong>on</strong> backward tracking of the activity. In other words, after the finished sessi<strong>on</strong><br />

the actor could be asked about the satisfacti<strong>on</strong> and reas<strong>on</strong>s for it. Such interviews could make the<br />

implicit objectives visible for both the actor interviewed and the interviewer.<br />

References<br />

Andresen, Jan L.: A Framework for Selecting IT Evaluati<strong>on</strong> Method – in the C<strong>on</strong>text of C<strong>on</strong>structi<strong>on</strong>. Danmarks<br />

tekniske universitet. 2001. ISBN 87-7877-069-6. PhD Thesis.<br />

Baillette, Paméla and Kimble, Chris. “The C<strong>on</strong>cept of Appropriati<strong>on</strong> as a Heuristic for C<strong>on</strong>ceptualising the<br />

Relati<strong>on</strong>ship between Technology, People and Organisati<strong>on</strong>s”. http://arxiv.org/pdf/0804.2847.pdf (reviewed<br />

April 22, 2012)<br />

Lahtiranta, Janne and Nurminen, Markku I. (2012) “PHR Revisi<strong>on</strong>ed – navigating in the pers<strong>on</strong>al health space”.<br />

Manuscript delivered for publicati<strong>on</strong>.<br />

Lyytinen, Kalle (1988) “Expectati<strong>on</strong> Failure C<strong>on</strong>cept and Systems Analysts’ View of Informati<strong>on</strong> System Failures:<br />

Results of an Exploratory Study”, Informati<strong>on</strong> & <strong>Management</strong>, Vol 14, pp 45-56.<br />

Schmidt, Kjeld and Bann<strong>on</strong>, Liam (1992):Taking CSCW Seriously: Supporting Articulati<strong>on</strong> Work, Computer<br />

Supported Cooperative Work, Vol 1, No 1-2, pp. 7-40.<br />

Stringer, Mark, Halloran, John, Hornecker, Eva and Fitzpatrick, Geraldine. “Situating Ubiquitous Computing in<br />

Everyday Life: Some Useful Strategies.<br />

Williams<strong>on</strong>, Oliver E. (1981):The Ec<strong>on</strong>omics of Organizati<strong>on</strong>: The Transacti<strong>on</strong> Cost Approach, The American<br />

journal of Sociology, Vol 87, No 2, pp.233-<br />

http://www.informatics.sussex.ac.uk/research/groups/interact/publicati<strong>on</strong>s/stringer_ubicomp05.pdf (reviewed April<br />

22, 2012)<br />

225


Infusi<strong>on</strong> of Mobile Health Systems in the NHS: An<br />

Empirical Study<br />

Yv<strong>on</strong>ne O’ C<strong>on</strong>nor 1 , John O’ D<strong>on</strong>oghue 2 and Phillip O’ Reilly 1<br />

1<br />

Department of Accounting, Finance and Informati<strong>on</strong> Systems, University<br />

College Cork, Ireland<br />

2<br />

Health Informati<strong>on</strong> Systems Research Centre, University College Cork, Ireland<br />

y.c.oc<strong>on</strong>nor@umail.ucc.ie<br />

john.od<strong>on</strong>oghue@ucc.ie<br />

phillip.oreilly@ucc.ie<br />

Abstract: Frequently criticised as a technological laggard, the healthcare industry is now beginning to appreciate<br />

the benefits which can be obtained from adopting Mobile Health Systems at the point-of-care. As a result,<br />

healthcare organisati<strong>on</strong>s are investing heavily in mobile health initiatives with the expectati<strong>on</strong> that individual users<br />

will employ the system to enhance performance. However, researchers argue that such benefits can <strong>on</strong>ly be fully<br />

realised if the technological innovati<strong>on</strong> is infused within an individual’s work practice. A synopsis of the ‘state of<br />

the field’ in mobile system implementati<strong>on</strong> research reveals that little is known <strong>on</strong> Mobile Health Systems<br />

infusi<strong>on</strong>. Infusi<strong>on</strong> is a distinctive feature in the Cooper and Zmud (1990) model, which reflects the extent to which<br />

a technological innovati<strong>on</strong> is fully embedded in an individual’s work system through comprehensive and<br />

integrative use. However a review of extant literature reveals that infusi<strong>on</strong> is inc<strong>on</strong>sistently defined and under<br />

investigated with a lack of literature focusing <strong>on</strong> Mobile Health Systems infusi<strong>on</strong>. This paper makes a number of<br />

c<strong>on</strong>tributi<strong>on</strong>s to the literature. It provides a comprehensive definiti<strong>on</strong> of infusi<strong>on</strong> and presents a c<strong>on</strong>ceptual model<br />

exploring infusi<strong>on</strong> of Mobile Health Systems. Through an exploratory study of Mobile Health Systems<br />

implementati<strong>on</strong> in Britain’s Nati<strong>on</strong>al Health Service, the presented model is empirically investigated. By<br />

identifying and highlighting issues affecting infusi<strong>on</strong>, future research efforts can focus <strong>on</strong> how such issues can be<br />

overcome. The paper c<strong>on</strong>cludes with a checklist of critical success factors which healthcare organisati<strong>on</strong>s should<br />

c<strong>on</strong>sider in order to successfully infuse Mobile Health Systems within their organisati<strong>on</strong>.<br />

Keywords: mobile health, infusi<strong>on</strong>, critical success factors, NHS<br />

1. Introducti<strong>on</strong> and theoretical grounding<br />

Over the last decade the applicati<strong>on</strong> of Informati<strong>on</strong> Technology (IT) in health care has grown greatly<br />

and its potential to improve effectiveness and efficiency has been recognised by governments’<br />

globally (Institute of Medicine, 2001). This is reflected in organisati<strong>on</strong>s worldwide investing heavily in<br />

the implementati<strong>on</strong> of technological innovati<strong>on</strong>s. For example, engagements in Swedish e-health<br />

initiatives cost the healthcare sector approximately €700 milli<strong>on</strong> annually (Ministry of Health and<br />

Social Affairs, 2010). Recent developments in healthcare have witnessed the emergence of<br />

ubiquitous computing, namely Mobile Health Systems (MHS). MHS is defined for the purpose of this<br />

paper as any mobile handheld device running medical applicati<strong>on</strong>s which are used as part of clinical<br />

practice. The rise in implementing Mobile Health Systems is reflected in the marketplace whereby the<br />

m-Health industry is valued between $50 billi<strong>on</strong> and $60 billi<strong>on</strong> globally (McKinsey, 2010).<br />

It is well established in the Informati<strong>on</strong> Systems (IS) literature that l<strong>on</strong>g-term success of a<br />

technological innovati<strong>on</strong> depends up<strong>on</strong> its adopti<strong>on</strong> and c<strong>on</strong>tinued use (Bhattacherjee, 2001).<br />

However, it is argued (Cooper and Zmud, 1990) that the true potential of technological innovati<strong>on</strong>s<br />

can <strong>on</strong>ly be achieved through infusi<strong>on</strong>. Yet, despite substantial research in adopti<strong>on</strong> and c<strong>on</strong>tinued<br />

use of MHS there remains a dearth of literature focusing <strong>on</strong> how infusi<strong>on</strong> can be achieved by<br />

individuals (O’ C<strong>on</strong>nor et al., 2012).<br />

In order to address this gap in literature this paper explores individual infusi<strong>on</strong> in order to identify and<br />

highlight issues affecting infusi<strong>on</strong> and to present a checklist of critical success factors which<br />

healthcare organisati<strong>on</strong>s should c<strong>on</strong>sider in order to successfully infuse MHS. The paper is structured<br />

as follows. MHS infusi<strong>on</strong> is discussed (secti<strong>on</strong> 1.1) whereby a comprehensive definiti<strong>on</strong> for the<br />

c<strong>on</strong>cept of infusi<strong>on</strong> is presented. The subsequent secti<strong>on</strong> (secti<strong>on</strong> 2) focuses <strong>on</strong> an Individual Mobile<br />

Health Infusi<strong>on</strong> Model. This c<strong>on</strong>ceptual model, which draws up<strong>on</strong> and extends extant literature, is<br />

operati<strong>on</strong>alised using a case study approach (secti<strong>on</strong> 3). Secti<strong>on</strong> 4 presents the findings leading to a<br />

seven critical success factors for individual infusi<strong>on</strong> of MHS. Secti<strong>on</strong> 5 presents the key implicati<strong>on</strong>s<br />

for theory and practice of this study and discusses the potential for future research within individual mhealth<br />

infusi<strong>on</strong>.<br />

226


Yv<strong>on</strong>ne O’ C<strong>on</strong>nor, John O’ D<strong>on</strong>oghue and Phillip O’ Reilly<br />

1.1 Mobile health system infusi<strong>on</strong>: Definiti<strong>on</strong><br />

Cooper and Zmud (1990) proposed a six phase model of IT implementati<strong>on</strong>. These stages include<br />

initiati<strong>on</strong>, adopti<strong>on</strong>, adaptati<strong>on</strong>, acceptance, rountizati<strong>on</strong> and infusi<strong>on</strong>. As a dearth of research exists<br />

which focuses <strong>on</strong> MHS infusi<strong>on</strong> this study primarily focuses <strong>on</strong> the final phase of the Cooper and<br />

Zmud (1990) model, namely infusi<strong>on</strong>.<br />

The c<strong>on</strong>cept of IT infusi<strong>on</strong> has being studied by numerous authors at various levels of analysis in<br />

diverse academic disciplines. Although renowned as the final stage in the Cooper and Zmud’s (1990)<br />

diffusi<strong>on</strong> model, there exists a lack of evidence as to the existence of a comprehensive definiti<strong>on</strong> of IT<br />

infusi<strong>on</strong>. Prior studies have defined IT infusi<strong>on</strong> in two levels: organisati<strong>on</strong>al and individual. Initially,<br />

when the c<strong>on</strong>cept of IT infusi<strong>on</strong> emerged in IS literature it was studied by many scholars (for example,<br />

Cooper and Zmud, 1990; Zmud and Apple, 1992; Saga 1994) at the organisati<strong>on</strong>al level. As a result it<br />

was often defined as “increased organisati<strong>on</strong>al effectiveness … obtained by using IT applicati<strong>on</strong> to its<br />

fullest potential” (Cooper and Zmud, 1990, pp. 124-125). Others have defined infusi<strong>on</strong> as “the degree<br />

to which IT has penetrated a company in terms of importance, impact, or significance” (Sullivan,<br />

1985) through the integrati<strong>on</strong> of technology with existing business processes (Eder and Igbaria, 2001,<br />

p. 234) whereby “the extent to which an innovati<strong>on</strong> is used completely and effectively and improves<br />

the organisati<strong>on</strong>’s performance” (Wynekoop and Senn, 1992).<br />

It is argued (Fadel, 2006) that organisati<strong>on</strong>al infusi<strong>on</strong> of any technological innovati<strong>on</strong> can <strong>on</strong>ly be<br />

achieved as individual’s infuse the technology into their own work practices. This is further reinforced<br />

by Sundaram et al., (2007) who argue before organisati<strong>on</strong>s can optimise Informati<strong>on</strong> Systems<br />

potential it should first optimise the potential of individual users. This rati<strong>on</strong>ale led to the modificati<strong>on</strong><br />

of existing definiti<strong>on</strong>s of infusi<strong>on</strong> in extant literature to reflect the individual user and not the<br />

organisati<strong>on</strong>. As a result, infusi<strong>on</strong> at an individual level is defined by Fadel (2006) as the “extent to<br />

which an informati<strong>on</strong> system is used completely and effectively and improves the individual’s<br />

performance”. Similarly, J<strong>on</strong>es et al., (2002) defines individual IT infusi<strong>on</strong> as the extent to which an<br />

individual user (a salespers<strong>on</strong> in their study) uses technology (Sales Force Automati<strong>on</strong>) to its “fullest<br />

extent to enhance their productivity”. A careful examinati<strong>on</strong> of these definiti<strong>on</strong>s suggests that<br />

comprehensive, integrative and inclusive use is the defining features of IT infusi<strong>on</strong> (Yu et al., 2009, O’<br />

C<strong>on</strong>nor et al., 2011). Therefore as this study investigates infusi<strong>on</strong> at an individual level it is defined<br />

herein as: “the degree to which individual users employs the full potential of a mobile health system<br />

within their work practices through comprehensive and integrated use” (adapted from O’ C<strong>on</strong>nor et<br />

al., 2012).<br />

The remainder of this paper is structured as follows. A model of determinants associated with MHS<br />

infusi<strong>on</strong> is presented (secti<strong>on</strong> 2). This Individual Mobile Health Infusi<strong>on</strong> Model, which draws up<strong>on</strong> and<br />

extends extant literature, is utilised as the basis for identifying technological issues impacting infusi<strong>on</strong><br />

(Secti<strong>on</strong> 4). Seven critical success factors are presented and described for healthcare practiti<strong>on</strong>ers to<br />

follow if they wish to infuse MHS are part of their work practices. Secti<strong>on</strong> 5 presents the key<br />

implicati<strong>on</strong>s for theory and practice of this study and discusses the potential for future research within<br />

individual MHS infusi<strong>on</strong>.<br />

2. Mobile health system infusi<strong>on</strong>: A c<strong>on</strong>ceptual model<br />

In order to enhance studies <strong>on</strong> IT infusi<strong>on</strong> many researchers (J<strong>on</strong>es et al., 2002; Wang and Hsieh,<br />

2006; Hsieh and Wang, 2007; Ramamurthy et al., 2008; Ng and Kim, 2009; Wu and Subramaniam,<br />

2009) turn to existing theories/models in the IS field to identify antecedents to infusi<strong>on</strong>. However,<br />

analysis of existing infusi<strong>on</strong> models revealed their unsuitability for investigating MHS infusi<strong>on</strong> am<strong>on</strong>g<br />

individuals, with such models primarily focused <strong>on</strong> infusi<strong>on</strong> of stati<strong>on</strong>ary technologies.<br />

Acknowledging that MHS differ from their static counterparts, O’ C<strong>on</strong>nor et al., (2012) proposed an<br />

‘Individual Mobile Health Infusi<strong>on</strong> (IMHI)’ model (Figure 1). These authors argue that successful<br />

infusi<strong>on</strong> is determined by three characteristics namely, technology, user and task characteristics.<br />

However, this paper focuses <strong>on</strong> three dimensi<strong>on</strong>s (i.e. Availability, Maturity and Portability) associated<br />

with technological characteristics. Given the important role that healthcare technology has <strong>on</strong> the<br />

delivery of quality healthcare services (S-Mohamadali and Garibaldi, 2012), it is important to<br />

investigate technology characteristics. The authors recognise that there are a number of other<br />

categories presented in the c<strong>on</strong>ceptual model which could have been explored. However, at an<br />

227


Yv<strong>on</strong>ne O’ C<strong>on</strong>nor, John O’ D<strong>on</strong>oghue and Phillip O’ Reilly<br />

individual level, infusi<strong>on</strong> is not likely to occur unless the user of the technology engages with the<br />

technology (Fadel, 2007).<br />

Figure 1: Individual mobile health infusi<strong>on</strong> (Source: O’ C<strong>on</strong>nor et al., 2012)<br />

Technology characteristics refer to specific features, functi<strong>on</strong>ality, or usability of a technology that can<br />

affect its infusi<strong>on</strong> by target users (adapted from Agarwal and Venkatesh, 2002). This characteristic<br />

has the following dimensi<strong>on</strong>s; availability, maturity and portability. Availability is the ability of<br />

accessing mobile health system when required. Maturity” implies the existence of a level of system<br />

quality that is perceived as satisfactory (Triandis, 1980) and the perceived need for system<br />

improvement (Gebauer, 2008) by the user. Therefore, it is c<strong>on</strong>ceptualised as the perceived need for<br />

system improvements by an individual (Gebauer, 2008). Portability refers to the degree of ease<br />

associated with transporting the mobile health system (Hoehle and Scornavacca 2008).<br />

3. Research methodology<br />

The objective of this research is twofold; (1) To identify and highlight issues affecting infusi<strong>on</strong> and (2)<br />

to present a checklist of critical success factors which healthcare organisati<strong>on</strong>s should c<strong>on</strong>sider in<br />

order to successfully infuse MHS. This is achieved through an exploratory study of MHS infusi<strong>on</strong> in<br />

Britain’s Nati<strong>on</strong>al Health Service, whereby the c<strong>on</strong>ceptual model highlighted previously is empirically<br />

investigated. Given the exploratory nature of this study, case study methods are presented as an<br />

appropriate approach in which to investigate the research objective. A single qualitative case study<br />

method is favourable given the research objective and current gap in the literature. Marshall and<br />

Rossman (1989) indicate that when the state of knowledge in a field is at an early stage of<br />

investigati<strong>on</strong>, a need exists for the research purpose to focus <strong>on</strong> ‘discovery’, and be ‘exploratory’ in<br />

nature. Galliers (1992) states that for such an exploratory approach a case study is a valid research<br />

method. The case study approach enables the researcher to investigate and capture the reality of the<br />

phenomen<strong>on</strong> (Yin, 1994).<br />

The Nati<strong>on</strong>al Health Service case was chosen as it represents a critical case with regard to<br />

understanding determinants of infusi<strong>on</strong> of MHS in a healthcare envir<strong>on</strong>ment. Data was gathered over<br />

a <strong>on</strong>e m<strong>on</strong>th period in October 2011. University Hospital Birmingham Nati<strong>on</strong>al Health Service<br />

Foundati<strong>on</strong> Trust (UHBFT) is <strong>on</strong>e of the most-c<strong>on</strong>sistently highest performing trusts in the Nati<strong>on</strong>al<br />

228


Yv<strong>on</strong>ne O’ C<strong>on</strong>nor, John O’ D<strong>on</strong>oghue and Phillip O’ Reilly<br />

Health Service and has been rated "excellent" for quality of clinical and n<strong>on</strong>-clinical services by the<br />

Healthcare Commissi<strong>on</strong>. UHBFT began using “tablet technology” ten years ago and currently utilises<br />

over 500 tablets in operati<strong>on</strong> within the Trust. Over ten hours of interviews were c<strong>on</strong>ducted <strong>on</strong>site with<br />

a broad spectrum of healthcare practiti<strong>on</strong>ers ranging from clinical lead in pharmacology, nurses, PICS<br />

(Prescribing Informati<strong>on</strong> and Communicati<strong>on</strong> System) training pers<strong>on</strong>nel to pharmacist technicians.<br />

Applying case study techniques, primary sources of empirical data c<strong>on</strong>sisted of interviews and the<br />

collecti<strong>on</strong> of numerous documents pertaining to MHS in the Nati<strong>on</strong>al Health Service. The MHS used<br />

in UHBFT included a Mobile Clinical Assistant (MCA) running Prescribing Informati<strong>on</strong> and<br />

Communicati<strong>on</strong> Systems.<br />

4. Findings<br />

This secti<strong>on</strong> identifies technological issues affecting infusi<strong>on</strong> of MHS and proposes a checklist of<br />

critical success factors which healthcare organisati<strong>on</strong> should c<strong>on</strong>sider in order to successfully infuse<br />

MHS within their organisati<strong>on</strong>. Healthcare practiti<strong>on</strong>ers indicate that technological dimensi<strong>on</strong>s such as<br />

availability, technology maturity and portability of MHS are pertinent for individual infusi<strong>on</strong>. Each<br />

dimensi<strong>on</strong> is discussed subsequently in more detail.<br />

4.1 Issues affecting infusi<strong>on</strong> - availability<br />

MHS are expensive (approximately £2,000 each), thus “it is a relatively large investment in MCA's”.<br />

As a result, some healthcare practiti<strong>on</strong>ers are required to share the equipment. Although most wards<br />

within the hospital were assigned 15 MHS there were often reports that some were missing from a<br />

ward. This issue is reflected in comments by various healthcare practiti<strong>on</strong>ers who stated that “there<br />

have been problems with wandering tablet PCs”. “I d<strong>on</strong>’t know where they go. I did a ward last week<br />

that should have had 15 and they had 7”. It was revealed that some healthcare practiti<strong>on</strong>ers who work<br />

<strong>on</strong> various wards carry <strong>on</strong>e MCA to different wards. As a result, some healthcare practiti<strong>on</strong>ers had to<br />

find an available Mobile Health System to work with, which was c<strong>on</strong>sidered as a “time c<strong>on</strong>suming<br />

activity”.<br />

Moreover, there are high reports of malfuncti<strong>on</strong>s associated with MCAs. Our findings reveal that this<br />

issue is more a social issue rather than a technological issue. This is exemplified in a comment<br />

revealed by <strong>on</strong>e medical practiti<strong>on</strong>er who states “we have more out of acti<strong>on</strong> because people haven’t<br />

bothered to report any problems with battery life or stylus missing”. While observing individual users<br />

of MCAs it became apparent that the styluses were being removed by a small minority of healthcare<br />

practiti<strong>on</strong>ers and stored in their pockets. As a result, other users would have access to the MCA but<br />

would have no method of inputting data thus, making the device redundant. As revealed in the<br />

previous comments there are also c<strong>on</strong>cerns about battery performance. Similarly, this is a social<br />

issue rather than a technical issue. Findings exemplify that the MCA “does not be put into the charger<br />

correctly”<br />

and that no <strong>on</strong>e is willing to take accountability for the charging of such devices as <strong>on</strong>e<br />

interviewee<br />

revealed “they are not a priority for the nurses, they are flying around all day”.<br />

MHS are ubiquitous in nature; therefore, such technologies require a stable underlying infrastructure<br />

(i.e. Wi-Fi) to operate. It was reported that initially “the Wi-Fi was a bit flaky with relatively few black<br />

spots”. As a result, this affected the use of MHS within the hospital. However, this issue was rectified<br />

and the majority of individuals interviewed agree that the Wi-Fi envir<strong>on</strong>ment is currently stable with<br />

“little than .07% of downtime running PICS (Prescribing Informati<strong>on</strong> and Communicati<strong>on</strong> Systems)<br />

over the last 8 years”.<br />

Moreover, in a healthcare envir<strong>on</strong>ment mobile devices are often “locked down”. Locking down mobile<br />

devices refers to limited access to applicati<strong>on</strong> and feature use of mobile artefacts. Due to privacy and<br />

security c<strong>on</strong>cerns the management team decided to lock down some features of the MHS. Therefore,<br />

“users can’t use the MCA for other purposes”. As a result, this hindered the high infusi<strong>on</strong> of the mobile<br />

artefact by individual users as users were limited in terms of “exploring the system”. As a result, some<br />

interviewees stated that “I d<strong>on</strong>’t think we use them enough”. Noteworthy, however, certain individuals<br />

have access to a testing envir<strong>on</strong>ment whereby they were free to explore the features of the PICS<br />

applicati<strong>on</strong>. This is exemplified in comments made by various interviewees (nurse, pharmacists, and<br />

clinical lead in pharmacology). For example, <strong>on</strong>e pharmacist stated that he “explored using the<br />

training domain”.<br />

229


Yv<strong>on</strong>ne O’ C<strong>on</strong>nor, John O’ D<strong>on</strong>oghue and Phillip O’ Reilly<br />

4.2 Issues affecting infusi<strong>on</strong> – technology maturity<br />

It is evident from the last comment that the technology is in place within the healthcare organisati<strong>on</strong><br />

for a l<strong>on</strong>g period of time (over 10 years). However, some individuals within the organisati<strong>on</strong> indicated<br />

that there was a need for system improvements. Comments from <strong>on</strong>e pharmacist indicate the need to<br />

update the system to meet their daily work practices. According to this individual “the software we feel<br />

from a pharmacy point of view needs a lot of changing to it – a lot more up to date things <strong>on</strong> there for<br />

example”. Acknowledging that the system did initially meet her needs (“when we first started using the<br />

PICS, having the electr<strong>on</strong>ic prescribing, it was brilliant”) the interviewee stated that “the way we work<br />

has changed and will change again” and this is not reflected in the MHS. Although changes are<br />

frequently made to the MHS, analysis revealed that “the process to get anything changed takes time”.<br />

Although the Mobile Health Systems “identifies some of the changes which have been made” it is the<br />

resp<strong>on</strong>sibility of the user to “pick up that they have changed”. These finding reveals the need for<br />

adapting the MHS to accommodate changing work practices and the need for change management.<br />

4.3 Issues affecting infusi<strong>on</strong> - portability<br />

In a healthcare envir<strong>on</strong>ment clinical work is highly mobile. Health professi<strong>on</strong>als move frequently<br />

am<strong>on</strong>g wards, clinics, offices, and other locati<strong>on</strong>s and require patient-related informati<strong>on</strong> at each of<br />

these locati<strong>on</strong>s. Comments from various healthcare practiti<strong>on</strong>ers indicate that the erg<strong>on</strong>omics<br />

associated with MHS is a c<strong>on</strong>cern. MHS must be designed to accommodate an individual’s work<br />

practices. Some people (junior doctors, pharmacists and nurses) feel that the device itself is “too<br />

unwieldy - it is too heavy” and this is having a negative impact <strong>on</strong> existing users with complaints of<br />

having “problems with their neck and shoulders” and “causes a lot of wrist pain”. As a result, some<br />

healthcare practiti<strong>on</strong>ers resorted to the use of COWS (Computers-On-Wheels). However, due to ward<br />

space limitati<strong>on</strong> those users who resorted to the use of COWS were also restricted in terms of their<br />

use. These findings clearly indicate that the portability of such devices impacts their use.<br />

4.4 Critical success factors (CSF) for mobile health infusi<strong>on</strong> by individuals<br />

This secti<strong>on</strong> c<strong>on</strong>cludes by leveraging the discussi<strong>on</strong> presented in the previous secti<strong>on</strong> to derive a set<br />

of seven macro-CSFs which require particular attenti<strong>on</strong> from managers in order to ensure MHS are<br />

infused by individuals. These seven CSFs for individual infusi<strong>on</strong> of MHS are described below:<br />

CSF 1: Ensure the Mobile Health Systems adapts to the users work/task practices<br />

This ensures that users of MHS will c<strong>on</strong>tinue using the MHS. Evidence supports the fact that when<br />

MHS provide alternative or supplementary products or services, and little effort is required to learn<br />

new operati<strong>on</strong>s or behaviour change, users are likely to c<strong>on</strong>tinue using the systems (Chen and<br />

Adams, 2005). In our findings the majority of interviewees indicated that they did not have to change<br />

their work practices thus, were happy to utilise the MHS. N<strong>on</strong>etheless, it was also evident that work<br />

practices of certain individuals within the organisati<strong>on</strong> did change overtime. Initially, the MHS met the<br />

needs of these healthcare practiti<strong>on</strong>ers but eventually their needs changed but the MHS did not adapt<br />

to the changes within the users’ work/task practices. Therefore, MHS must be c<strong>on</strong>tinuously adapted<br />

to the users work/task practices.<br />

CSF 2: Establish Change <strong>Management</strong> Protocol<br />

This ensures that any changes made to the MHS will be communicated to the relevant parties. Any<br />

technological innovati<strong>on</strong> which has been in place for a l<strong>on</strong>g period of time comm<strong>on</strong>ly involves some<br />

element of change. Therefore, it is important to c<strong>on</strong>trol change. Having a dedicated team to promote<br />

and communicate changes to individual users is pertinent for the infusi<strong>on</strong> of work practices. Evidence<br />

postulates that communicati<strong>on</strong> between the dedicated team and individual users of mobile health<br />

systems is critical in c<strong>on</strong>trolling change management.<br />

CSF 3: Ensure the Mobile Health Systems is fit for purpose and a stable infrastructure exist<br />

This ensures that the MHS will be used in a comprehensive manner. In our findings interviewees<br />

stipulated that the erg<strong>on</strong>omics of the MHS was an issue. Some interviewees believed that the<br />

handheld mobile devices were heavy and thus, their portability was insufficient for daily use. As a<br />

result, they often resorted to COWS. Therefore in selecting hardware mobile devices, c<strong>on</strong>siderati<strong>on</strong><br />

should be given to those individuals who will be using the devices. Although selecting the hardware<br />

devices is pertinent, it is imperative that healthcare practiti<strong>on</strong>ers can access relevant patient data via<br />

the devices when required. Therefore, it is integral to have a stable, underlying Wi-Fi infrastructure in<br />

230


Yv<strong>on</strong>ne O’ C<strong>on</strong>nor, John O’ D<strong>on</strong>oghue and Phillip O’ Reilly<br />

place. Without a stable infrastructure the MHS cannot be used thus, impacting up<strong>on</strong> individuals’<br />

infusi<strong>on</strong> of the device.<br />

CSF 4: Train users of Mobile Health Systems to maintain the Mobile Health System<br />

This ensures that MHS will be available when required. It is evident from the findings that key issues<br />

surrounding physical attributes of the MHS (namely, poor battery life and limited device accessories<br />

i.e. stylus) restrict users from infusing mobile health systems as part of their clinical practice. The<br />

findings revealed that these issues were social rather than technical. Therefore, it is pertinent to<br />

provide adequate training to users <strong>on</strong> maintaining the MHS to overcome such issues.<br />

CSF 5: Ensure that there is a dedicated team to support Mobile Health Systems<br />

This ensures total commitment to the MHS at an organisati<strong>on</strong>al level. It is important to realise that<br />

infusi<strong>on</strong> of mobile artefacts will not occur overnight. In this case study, the use of MHS (i.e. MCA with<br />

PICS) was implemented initially “back in 1999” with some key players champi<strong>on</strong>ing the<br />

implementati<strong>on</strong>. According to the clinical lead in pharmacology having key players involved in the<br />

implementati<strong>on</strong> process is integral, “without that it would have been terribly difficult so that’s why it<br />

survived”. The reas<strong>on</strong>s underlying the success of the MCA running PICS to date is the establishment<br />

of a PICS development team. This team is resp<strong>on</strong>sible for organising weekly meetings with various<br />

medical practiti<strong>on</strong>er groups (for example, PICS steering group, PICS nursing group) to discuss any<br />

updates or amendments made to the electr<strong>on</strong>ic prescribing systems.<br />

CSF 6: Saturate the organisati<strong>on</strong> with Mobile Health Systems<br />

This ensures that individual users have the possibility to interact with the MHS. Evidence supports the<br />

fact that users who have a propensity to spend more time <strong>on</strong> a system will learn new ways of<br />

exploiting the system’s capabilities or become more adept at “discovering” more efficient ways of<br />

using systems outside of their original use (Jain and Kanungo, 2006). However, some healthcare<br />

organisati<strong>on</strong>s do not have a sufficient amount of MHS available to the end user. It is imperative<br />

therefore to have enough MHS for end users to interact with.<br />

CSF 7: Promote comprehensive and integrative use of Mobile Health Systems by creating a safe<br />

envir<strong>on</strong>ment to exploit the system.<br />

This ensures that users go bey<strong>on</strong>d routine and standardised usage of MHS. As a result, these<br />

individuals achieve a higher level of usage that may allow them to exploit the fullest potential of the<br />

system. Evidence exist which highlight that individual users are reluctant to explore any system in an<br />

healthcare organisati<strong>on</strong> due to the ill-affects their acti<strong>on</strong>s may have <strong>on</strong> the delivery of healthcare<br />

services to patients at the point-of-care. The additi<strong>on</strong> of a ‘safe’ envir<strong>on</strong>ment (e.g. training<br />

envir<strong>on</strong>ment with dummy results) where individuals can go bey<strong>on</strong>d routine and standardised should<br />

be established. The existence of a ‘safe’ envir<strong>on</strong>ment is critical to ensuring that individuals have the<br />

opportunity to explore the system.<br />

5. C<strong>on</strong>cluding remarks: Implicati<strong>on</strong>s and further research<br />

Achieving individual infusi<strong>on</strong> of any technological innovati<strong>on</strong> is a difficult process. As a result, this<br />

paper focuses <strong>on</strong> individual infusi<strong>on</strong> of Mobile Health Systems (MHS). This paper describes the<br />

issues associated with an individual’s infusi<strong>on</strong> of MHS. By identifying and highlighting issues affecting<br />

infusi<strong>on</strong>, future research efforts may focus <strong>on</strong> how such issues can be overcome.<br />

This study offers several implicati<strong>on</strong>s for both theory and practice. The first c<strong>on</strong>tributi<strong>on</strong> to theory is<br />

the presentati<strong>on</strong> of a comprehensive definiti<strong>on</strong> for ‘infusi<strong>on</strong>’. Analysis of the literature pertaining to<br />

infusi<strong>on</strong> revealed inc<strong>on</strong>sistencies for what c<strong>on</strong>stitutes the term ‘infusi<strong>on</strong>’. Based <strong>on</strong> this analysis,<br />

infusi<strong>on</strong> refers to ““the degree to which individual users employs the full potential of a mobile health<br />

system within their work practices through comprehensive and integrated use”. Sec<strong>on</strong>dly, this paper<br />

is am<strong>on</strong>g the first to explore MHS infusi<strong>on</strong> by individuals, more specifically in a healthcare<br />

organisati<strong>on</strong>. In doing so, the research findings presented in this paper c<strong>on</strong>tribute to theory<br />

development in IS by adding to the current, limited understanding of individual infusi<strong>on</strong> of MHS.<br />

This study has potentially significant implicati<strong>on</strong>s for organisati<strong>on</strong>s looking to invest in m-health<br />

technologies and for those seeking to infuse MHS as part of their daily work practices. A set of seven<br />

macro-CSFs are derived from our findings which if followed, theoretically, should result in the<br />

successful infusi<strong>on</strong> of MHS at the individual level. Whilst this study has c<strong>on</strong>tributed to the domain of<br />

individual infusi<strong>on</strong> of MHS more research it has its limitati<strong>on</strong>s and requires further research. Firstly,<br />

231


Yv<strong>on</strong>ne O’ C<strong>on</strong>nor, John O’ D<strong>on</strong>oghue and Phillip O’ Reilly<br />

the findings are derived from the use of a single case study. It is argued that single case studies limit<br />

the generalizability of the results. Sec<strong>on</strong>dly, the seven derived CSFs are primarily associated with<br />

technological characteristics affecting infusi<strong>on</strong>. As a result, the seven CSFs are not an exhaustive list<br />

of CSFs for achieving individual infusi<strong>on</strong>. Further research is required to investigate CSFs associated<br />

with user and task characteristics. It is <strong>on</strong>ly then that a full, exhaustive list of CSFs can be presented<br />

to management to ensure individual infusi<strong>on</strong> of MHS is achieved.<br />

Acknowledgements<br />

This research was partially funded by Business Informati<strong>on</strong> Systems, <str<strong>on</strong>g>C<strong>on</strong>ference</str<strong>on</strong>g> Travel Support<br />

Scheme, University College Cork, Health Informati<strong>on</strong> Systems Research Centre (HISRC), and by the<br />

Science Foundati<strong>on</strong> Ireland (SFI) SFI"11/RFP.1/CMS/3338.<br />

References<br />

Agarwal, R. and V. Venkatesh (2002) "Assessing a Firm’s Web Presence: A Heuristic Evaluati<strong>on</strong> Procedure for<br />

the Measurement of Usability", Informati<strong>on</strong> Systems Research, 13(2): 169.<br />

Chen, JJ. And C. Adams (2005) “User Acceptance of Mobile Payments: A Theoretical Model for Mobile<br />

Payments”, Proceedings of the Fifth Internati<strong>on</strong>al <str<strong>on</strong>g>C<strong>on</strong>ference</str<strong>on</strong>g> <strong>on</strong> Electr<strong>on</strong>ic Business (ICEB), H<strong>on</strong>g K<strong>on</strong>g,<br />

December 5-9, 2005<br />

Cooper RB. And Zmud, RW. (1990), “Informati<strong>on</strong> Technology Implementati<strong>on</strong> Research: A Technology Diffusi<strong>on</strong><br />

Approach”, <strong>Management</strong> Science, 36(2), pp. 123–139.<br />

Eder, L. B. and M. Igbaria (2001) "Determinants of intranet diffusi<strong>on</strong> and infusi<strong>on</strong>", Omega 29(3): 233-242.<br />

Fadel, Kelly, (2006) "Individual Infusi<strong>on</strong> of Informati<strong>on</strong> Systems: The Role of Adaptati<strong>on</strong> and Individual<br />

Cogniti<strong>on</strong>s". AMCIS 2006, Proceedings Paper 38.<br />

Fadel, K.J. (2007). “Infusi<strong>on</strong> of Informati<strong>on</strong> Systems: The Role of Adaptati<strong>on</strong> and Individual Cogniti<strong>on</strong>s”, Ph.D.<br />

Dissertati<strong>on</strong>, The University of Ariz<strong>on</strong>a Graduate College, 2007.<br />

Gebauer, J. (2008) "User requirements of Mobile Technology: A summary of Research Results." Informati<strong>on</strong>,<br />

Knowledge, Systems <strong>Management</strong> 7(1): 101-119.<br />

Hoehle, H. and E. Scornavacca (2008) “Unveiling Experts Percepti<strong>on</strong>s towards the Characteristics and Value<br />

Propositi<strong>on</strong>s of Mobile Informati<strong>on</strong> Systems”, 7th Internati<strong>on</strong>al <str<strong>on</strong>g>C<strong>on</strong>ference</str<strong>on</strong>g> <strong>on</strong> Mobile Business, IEEE, pp.<br />

334 – 343<br />

Hsieh, J. J. P. A. and W. Wang (2007) "Explaining Employees' Extended Use of Complex Informati<strong>on</strong> Systems",<br />

<str<strong>on</strong>g>European</str<strong>on</strong>g> Journal of Informati<strong>on</strong> Systems 16(3): 216-227.<br />

Institute of Medicine (2001) Crossing the Quality Chasm: A New Health System for the 21st Century.<br />

Washingt<strong>on</strong>, DC: Nati<strong>on</strong>al Academy Press.<br />

Jain, V. and S. Kanungo, (2006) “IS-enabled performance improvement at the individual level: evidence of<br />

complementarity,” Proceedings of the 2006 ACM SIGMIS CPR c<strong>on</strong>ference <strong>on</strong> computer pers<strong>on</strong>nel<br />

research: Forty four years of computer pers<strong>on</strong>nel research: achievements, challenges & the future, ACM<br />

New York, NY, USA,<br />

J<strong>on</strong>es, E., Sundaram, S., and Chin, W. (2002) "Factors Leading to Sales Force Automati<strong>on</strong> Use: A L<strong>on</strong>gitudinal<br />

Analysis," Journal of Pers<strong>on</strong>al Selling & Sales <strong>Management</strong> 22(3), pp. 145-156.<br />

McKinsy and GSMA (2010), “mhealth; A new Visi<strong>on</strong> for m-health,” McKinsy and Company, Inc. and GSMA, 2010.<br />

Available:<br />

(http://www.gsmaembeddedmobile.com/upload/resources/files/GSMA%20McKinsey%20mHealth_report.pdf<br />

)<br />

Ministry of Health and Social Affairs (2010) Nati<strong>on</strong>al eHealth – the strategy for accessible and secure informati<strong>on</strong><br />

in health and social care Available <strong>on</strong>line: http://www.sweden.gov.se/c<strong>on</strong>tent/1/c6/16/79/85/8d4e6161.pdf<br />

Ng, E.H. and H.W. Kim, (2009) “Investigating Informati<strong>on</strong> Systems Infusi<strong>on</strong> and the Moderating Role of Habit: A<br />

User Empowerment Perspective”, Proceedings of Internati<strong>on</strong>al <str<strong>on</strong>g>C<strong>on</strong>ference</str<strong>on</strong>g> of Informati<strong>on</strong> Systems, ICIS<br />

2009 Proceedings. Paper 137<br />

O'C<strong>on</strong>nor, Y., J. O'D<strong>on</strong>oghue, and P. O’ Reilly (2011) “Understanding Mobile Technology Post-Adopti<strong>on</strong><br />

Behaviour: Impact up<strong>on</strong> Knowledge Creati<strong>on</strong> and Individual Performance”, Tenth Internati<strong>on</strong>al <str<strong>on</strong>g>C<strong>on</strong>ference</str<strong>on</strong>g><br />

<strong>on</strong> Mobile Business (ICMB), 2011.<br />

O'C<strong>on</strong>nor, Y., PJ. O’Raghailligh, and J. O’ D<strong>on</strong>oghue. (2012). “Individual Infusi<strong>on</strong> of M-Health Technologies:<br />

Determinants and Outcomes”, ECIS 2012, in Press<br />

Ramamurthy, K., A. Sen and A.P. Sinha. (2008). “Data Warehousing Infusi<strong>on</strong> and Organizati<strong>on</strong>al Effectiveness.<br />

Systems, Man and Cybernetics, Part A: Systems and Humans”, IEEE Transacti<strong>on</strong>s, 38(4): 976-994.<br />

S-Mohamadali, N. A. K. and J. M. Garibaldi (2012) “Understanding and Addressing the 'Fit' Between User,<br />

Technology and Organizati<strong>on</strong> in Evaluating User Acceptance of Healthcare Technology”, HealthInf 2012<br />

Internati<strong>on</strong>al <str<strong>on</strong>g>C<strong>on</strong>ference</str<strong>on</strong>g> <strong>on</strong> Health Informatics: 119-124.<br />

Saga, V. L. and R. W. Zmud (1994). "The nature and determinants of IT acceptance, routinizati<strong>on</strong> and infusi<strong>on</strong>",<br />

Diffusi<strong>on</strong> Transfer and Implementati<strong>on</strong> of Informati<strong>on</strong> Technology 45(A-45): 67-86.<br />

Sullivan, C. H. (1985) "Systems planning in the informati<strong>on</strong> age", Sloan <strong>Management</strong> Review 26(2): 3-12.<br />

Sundaram, S.Schwarz, A., J<strong>on</strong>es, E. and W. Chin (2007) “Technology use <strong>on</strong> the fr<strong>on</strong>t line: how informati<strong>on</strong><br />

technology enhances individual performance”, Journal of the Academy of Marketing Science, 35(1), p.101-<br />

112.<br />

232


Yv<strong>on</strong>ne O’ C<strong>on</strong>nor, John O’ D<strong>on</strong>oghue and Phillip O’ Reilly<br />

Triandis, H. C. (1980) “Values, Attitudes, and Interpers<strong>on</strong>al Behavior”, in H.E. Howe (ed.), Nebraska Symposium<br />

<strong>on</strong> Motivati<strong>on</strong>, 1979: Beliefs, Attitudes and Values, Lincoln, NE: University of Nebraska Press, pp. 195-259<br />

Wang, Wei and Hsieh, Po-An, (2006) "Bey<strong>on</strong>d Routine: Symbolic Adopti<strong>on</strong>, Extended Use, and Emergent Use of<br />

Complex Informati<strong>on</strong> Systems in the Mandatory Organizati<strong>on</strong>al C<strong>on</strong>text" (2006). ICIS 2006, Proceedings.<br />

Paper 48.<br />

Wu, X. and C. Subramaniam. (2009) “Understanding RFID Adopti<strong>on</strong> in Supply Chain: An Empirical Study”,<br />

Proceedings of Forty Sec<strong>on</strong>d Hawaii Internati<strong>on</strong>al <str<strong>on</strong>g>C<strong>on</strong>ference</str<strong>on</strong>g> <strong>on</strong> Systems Science, January 5-9, 2009.<br />

Wynekoop, J. L. and J. A. Senn (1992) “Case Implementati<strong>on</strong>: The Importance of Multiple Perspectives”,<br />

SIGCPR '92 Proceedings of the 1992 ACM SIGCPR c<strong>on</strong>ference <strong>on</strong> Computer pers<strong>on</strong>nel research, ACM.<br />

Yu, S., A. N. Mishra, et al. (2009) “IT Infusi<strong>on</strong> and its Performance Impacts: An Empirical Analysis of<br />

eProcurement in the Service Industry”, ICIS 2009 Proceedings. Paper 121.<br />

Zmud, R. W. and L. E. Apple (1992) "Measuring technology incorporati<strong>on</strong>/infusi<strong>on</strong>." Journal of Product Innovati<strong>on</strong><br />

<strong>Management</strong> 9(2): 148-155.<br />

233


An Exploratory Study of Innovati<strong>on</strong> Intermediati<strong>on</strong> in IS<br />

Educati<strong>on</strong><br />

Brian O’Flaherty and Joe Bogue<br />

Departments of Business Informati<strong>on</strong> Systems and Food Business and<br />

Development, University College Cork, Cork Ireland<br />

boflaherty@afis.ucc.ie<br />

j.bogue@ucc.ie<br />

Abstract: The importance of innovati<strong>on</strong> to ec<strong>on</strong>omies across the world has been widely documented and this<br />

has been particularly true in Irish terms in the development of a knowledge-based ec<strong>on</strong>omy (SSTI, 2006). One of<br />

the ways of facilitating an innovati<strong>on</strong> culture is through third level educati<strong>on</strong> and students taking innovati<strong>on</strong> and<br />

entrepreneurship programmes or modules. Innovati<strong>on</strong> educati<strong>on</strong> is a central part of third level educati<strong>on</strong> globally<br />

(O’Gorman and Fitzsim<strong>on</strong>s, 2007; Streeter and Jaquette, 2004) and manifests itself in different undergraduate<br />

and postgraduate levels and across different disciplines, such as, ICT, Engineering and Food. There is clear<br />

evidence of significant innovati<strong>on</strong> and entrepreneurship activity in Ireland (Co<strong>on</strong>ey and Murray, 2008) and<br />

worldwide (World Ec<strong>on</strong>omic Forum, 2009), yet there is limited research into the organisati<strong>on</strong> or process roles of<br />

participants in this area and even less cross disciplinary comparative reflecti<strong>on</strong>. The objective of this paper is to<br />

investigate the innovati<strong>on</strong> intermediati<strong>on</strong> role of IS student enterprise teams, identify the intermediati<strong>on</strong><br />

processes adopted, explore sources of innovati<strong>on</strong>, examine the practice and effectiveness of the process and<br />

c<strong>on</strong>sider the comparative cross disciplinary implicati<strong>on</strong>s in relati<strong>on</strong> to the food sector. Innovati<strong>on</strong> intermediati<strong>on</strong><br />

(Howells, 2006) is the theoretical sensitising lens that underpins this research study. This perspective is widely<br />

cited and has been applied to countless research areas, such as technology transfer, innovati<strong>on</strong> and networks.<br />

Innovati<strong>on</strong> intermediati<strong>on</strong> can be viewed as organisati<strong>on</strong>al roles, such as bridge builders, technology brokering,<br />

surrogate ties and diffusi<strong>on</strong> facilitati<strong>on</strong>. It can also refer to a range of functi<strong>on</strong>s in innovati<strong>on</strong> processes, such as<br />

scanning, knowledge creati<strong>on</strong>, testing, validati<strong>on</strong> and commercialisati<strong>on</strong>. The innovati<strong>on</strong> intermediati<strong>on</strong> model is<br />

therefore an ideal device for articulating this comparative case study research.<br />

Keywords: innovati<strong>on</strong> intermediati<strong>on</strong>, IS educati<strong>on</strong>, entrepreneurship, student enterprise<br />

1. Introducti<strong>on</strong><br />

This paper presents research from a study of IS student enterprise projects and looks at the<br />

intermediati<strong>on</strong> role they play in the innovati<strong>on</strong> process. In additi<strong>on</strong>, this intermediati<strong>on</strong> role is analysed<br />

in comparis<strong>on</strong> to similar student enterprise processes undertaken by Food Business students.<br />

1.1 Entrepreneurship and innovati<strong>on</strong><br />

The role of entrepreneurship educati<strong>on</strong> in the development of entrepreneurial societies across the<br />

world is central to ec<strong>on</strong>omic development. According to Hisrich et al. (2005) entrepreneurship has<br />

been endorsed by educati<strong>on</strong>al and entrepreneurial educati<strong>on</strong> and has never been so important to<br />

societies. Goodbody Ec<strong>on</strong>omic C<strong>on</strong>sultants (2002) found that the Irish Educati<strong>on</strong>al System was seen<br />

by entrepreneurs to have played a very limited role in practice and that there was very little direct<br />

focus <strong>on</strong> entrepreneurship and innovati<strong>on</strong> within the Irish Educati<strong>on</strong>al system. In additi<strong>on</strong>, Fitzsim<strong>on</strong>s<br />

and O’ Gorman (2005: 4) in the Global Entrepreneurship M<strong>on</strong>itor 2005 recommended that: “The<br />

educati<strong>on</strong> sector should be harnessed in a systematic way across all disciplines to increase<br />

entrepreneurial mindsets and to enhance the capacity of those who decide to become entrepreneurs”.<br />

Hisrich et al. (2005) noted that many universities across the United States had courses in<br />

entrepreneurship and that the courses could be found in liberal arts colleges, business schools and<br />

engineering schools. There are also universities in the US offering majors in entrepreneurship and<br />

entrepreneurial skills can be classified into three main categories: technical skills, management skills,<br />

and pers<strong>on</strong>al skills. They have noted this as a new phenomen<strong>on</strong> due to changes in technology,<br />

computers and competiti<strong>on</strong>. However, Hisrich et al. (2005) noted that few students in college think<br />

they will pursue entrepreneurship as their major life goal. However, the figure they noted had<br />

increased but that they generally did not start a business directly after their university studies (Hisrich<br />

et al., 2005). Educati<strong>on</strong> is also a central aspect entrepreneurship and c<strong>on</strong>trary to opini<strong>on</strong>,<br />

entrepreneurs are not less educated than the general populati<strong>on</strong> ((Hisrich et al., 2005). In fact, both<br />

male and females entrepreneurs are more educated than the general populace. This highlights the<br />

importance of entrepreneurship educati<strong>on</strong> and also the need for research of this nature.<br />

234


Brian O’Flaherty and Joe Bogue<br />

The importance of entrepreneurship in terms of the Irish ec<strong>on</strong>omy is well documented but this has not<br />

been mirrored in the educati<strong>on</strong> system. Fitzsim<strong>on</strong>s and O’ Gorman (2005) noted that there was a<br />

need for the development of the entrepreneurship agenda in the educati<strong>on</strong> system and that there was<br />

a need to develop creativity within students in terms of the innovati<strong>on</strong> process and to then link this<br />

creativity with the c<strong>on</strong>cept of entrepreneurship. They also suggested at third level that there should be<br />

more emphasis <strong>on</strong> developing entrepreneurship and innovati<strong>on</strong> modules across all degree<br />

programmes in science, technology and business studies (Fitzsim<strong>on</strong>s and O’ Gorman, 2005).<br />

1.2 Innovati<strong>on</strong> intermediati<strong>on</strong><br />

The role of intermediaries in entrepreneurship and the innovati<strong>on</strong> process is significant in terms of the<br />

role they play in facilitating the innovati<strong>on</strong> process and ultimately in the commercialisati<strong>on</strong> of new<br />

technologies. Howells (2006) noted that the different actors who perform a variety of tasks al<strong>on</strong>g the<br />

innovati<strong>on</strong> process are called ‘intermediaries’. Innovati<strong>on</strong> Intermediati<strong>on</strong> is defined as a role between<br />

creators and users of inventi<strong>on</strong>s (Hoppe and Ozdenoren, 2005). Howells (2006) defines the<br />

innovati<strong>on</strong> intermediary as “an organisati<strong>on</strong> or body that acts as an agent or broker in any aspect of<br />

the innovati<strong>on</strong> process between two or more parties.” Interest in the role of intermediaries has<br />

emerged in various interest areas, such as: 1) the technology transfer literature, 2) general innovati<strong>on</strong><br />

management of intermediati<strong>on</strong> and the organisati<strong>on</strong>s that deliver this functi<strong>on</strong>, 3) the innovati<strong>on</strong><br />

systems literature and 4) service and Knowledge Intensive Business Services (KIBS) firms (Howells,<br />

2006). Many intermediaries exist, such as: University Technology Transfer Offices, who assess the<br />

commercial value of technologies and seek investors with the resources and interest to further exploit<br />

the technology. Similarly, venture capitalists allocate resources to entrepreneurs, who take <strong>on</strong> an<br />

intermediary role typically in industries with high risk informati<strong>on</strong> gaps between technology and<br />

markets. Furthermore, innovati<strong>on</strong> intermediati<strong>on</strong> has been in existence for some time. Although,<br />

intermediaries can be traced back to ‘middlemen’ in the agricultural, wool and textiles industries of the<br />

sixteenth century, the role as it exists in the innovati<strong>on</strong> process has <strong>on</strong>ly being significantly recognised<br />

in recent years (Howells, 2006).<br />

The Innovati<strong>on</strong> Intermediati<strong>on</strong> literature has been applied in numerous ways and across many<br />

industry and domain areas (Howells, 2006). As a c<strong>on</strong>ceptual phenomen<strong>on</strong> it underpins research<br />

explaining quite an extensive range of inter-organisati<strong>on</strong>al structures, such as: innovati<strong>on</strong><br />

intermediaries and internet market places (Lichtenthaler and Ernst, 2008); knowledge intermediati<strong>on</strong><br />

between Universities and businesses (Yusuf, 2008); academic inventors as brokers (Liss<strong>on</strong>i, 2008);<br />

intermediaries in cross-industry innovati<strong>on</strong> processes (Gassmann et al., 2011); the role of incubators<br />

as intermediaries in knowledge transfer processes in product development in small technology -parks<br />

(Saari and Haapasalo, 2012); and intermediati<strong>on</strong> roles between Universities and businesses<br />

facilitating knowledge creati<strong>on</strong> in an innovati<strong>on</strong> system (Metcalfe, 2009).<br />

A number of authors have applied intermediati<strong>on</strong> to the agricultural and food business areas, with<br />

regi<strong>on</strong>al development outcomes, by c<strong>on</strong>sidering the bridging functi<strong>on</strong> between supply and demand<br />

side of agricultural knowledge infrastructures (Klerkx and Leeuwis, 2008), as well as, innovati<strong>on</strong><br />

brokers that orchestrate agri-food innovati<strong>on</strong> networks (Batterink et al., 2010). More specific to<br />

educati<strong>on</strong>, the Innovati<strong>on</strong> Intermediati<strong>on</strong> c<strong>on</strong>cepts are used to assess the effectiveness of a Global<br />

Access Program (GAP), a c<strong>on</strong>tinuing professi<strong>on</strong>al development support programme, which assists<br />

Finnish firms gain access to global markets (Dalziel and Parjanen, 2012). A study of the impact of<br />

entrepreneurship educati<strong>on</strong> in Higher Educati<strong>on</strong> Instituti<strong>on</strong>s (HEI) and Small to Medium Sized<br />

Enterprises (SME) recommends that more research is required into intermediaries, their network<br />

relati<strong>on</strong>ships and the impact <strong>on</strong> entrepreneurship educati<strong>on</strong> (Gord<strong>on</strong> et al., 2010). It is legitimate<br />

therefore to c<strong>on</strong>sider innovati<strong>on</strong> intermediati<strong>on</strong> activity in the c<strong>on</strong>text of this study of technology and<br />

food entrepreneurship educati<strong>on</strong>.<br />

Innovati<strong>on</strong> Intermediati<strong>on</strong> research has significant characteristics, namely there is an overlap between<br />

c<strong>on</strong>ceptual approaches, there is a distincti<strong>on</strong> between studies that focussed <strong>on</strong> intermediaries as<br />

organisati<strong>on</strong> and intermediati<strong>on</strong> as a process. However, there is a low level of cross referencing<br />

across the research and the explorati<strong>on</strong> of intermediaries in the innovati<strong>on</strong> process has not been wellgrounded<br />

theoretically (Howells, 2006). In synthesising the literature, Howells (2006) makes a<br />

distincti<strong>on</strong> between the perspectives of Intermediaries as organisati<strong>on</strong> and intermediati<strong>on</strong> as a<br />

process. It is legitimate to c<strong>on</strong>sider Student Enterprise Projects from the perspective of organisati<strong>on</strong><br />

roles (Howells, 2006), which is summarised in Table 1. Intermediati<strong>on</strong> is also c<strong>on</strong>sidered as an<br />

organisati<strong>on</strong>al role, within a network topology (Liss<strong>on</strong>i, 2008). A categorisati<strong>on</strong> of a new range of<br />

235


Brian O’Flaherty and Joe Bogue<br />

technology market intermediaries is described by Tietze (2010), which dem<strong>on</strong>strates how the<br />

innovati<strong>on</strong> intermediati<strong>on</strong> literature explains emerging new role insights.<br />

The intermediati<strong>on</strong> process as defined by Howells (2006) is a broad ranging typology that is too<br />

comprehensive for assessing Student Enterprise Projects within an academic envir<strong>on</strong>ment. This<br />

present study uses Howell’s role typology as a sensitising framework for examining the innovati<strong>on</strong> the<br />

intermediary roles of student teams. The research questi<strong>on</strong>s posed in this cross domain research<br />

are: 1) what are the intermediary roles of student enterprise teams and 2) what are the comparative<br />

characteristics of student enterprise projects?<br />

Table 1: Summary of intermediary roles (adapted from: Howells, 2006)<br />

Role Descripti<strong>on</strong><br />

Intermediaries Explores role of intermediary agencies support technology transfer to small firms.<br />

Examines the role of intermediaries in technology exploitati<strong>on</strong>.<br />

Role of intermediaries in effecting change within science networks and local<br />

collectives.<br />

Adapt soluti<strong>on</strong>s available in the market to the needs of the individual User.<br />

Public and private organizati<strong>on</strong>s that act as agents transferring technology between<br />

hosts and users.<br />

Role of missi<strong>on</strong> agencies in formulating research policy.<br />

Proactive role that certain types of service firms play as intermediaries within<br />

innovati<strong>on</strong> systems.<br />

Help orient the science system to socio-ec<strong>on</strong>omic objectives.<br />

Actors filling gaps in informati<strong>on</strong> and knowledge in industrial networks.<br />

Organizati<strong>on</strong>s that facilitate a recipient’s measurement of the intangible value of<br />

knowledge received.<br />

Third parties Pers<strong>on</strong>s or organizati<strong>on</strong>s that intervene in the adopti<strong>on</strong> decisi<strong>on</strong>s of others<br />

Brokers Agents facilitating the diffusi<strong>on</strong> of a social systems of new ideas from outside the<br />

system<br />

C<strong>on</strong>sultants as Role of independent c<strong>on</strong>sultants as bridge builders in the innovati<strong>on</strong> process<br />

bridge builders<br />

Bricoleurs Agents seeking to develop new applicati<strong>on</strong>s for new technologies outside their initial<br />

development field<br />

Superstructure Organizati<strong>on</strong>s that help to facilitate and coordinate the flow of informati<strong>on</strong> to<br />

organizati<strong>on</strong>s<br />

substructure firms<br />

Regi<strong>on</strong>al Provide ‘surrogate ties’ by serving as functi<strong>on</strong>al substitutes for a firm’s lack of ‘bridging<br />

instituti<strong>on</strong>s<br />

ties’ in a network<br />

Boundary<br />

Role of boundary organizati<strong>on</strong>s in technology transfer and ‘co-producti<strong>on</strong>’ of<br />

organizati<strong>on</strong>s<br />

Knowledge<br />

brokers<br />

2. Research methodology<br />

technology<br />

Agents that help innovati<strong>on</strong> by combining existing technologies in<br />

new ways<br />

The objectives of this research were: 1. to investigate the innovati<strong>on</strong> intermediati<strong>on</strong> role of Informati<strong>on</strong><br />

Systems (IS) student enterprise teams; and 2. to explore sources of innovati<strong>on</strong>, examine the practice<br />

and effectiveness of the process and c<strong>on</strong>sider the comparative cross disciplinary implicati<strong>on</strong>s in<br />

relati<strong>on</strong> to the food sector.<br />

The research methodology adopted was a comparative case study of two separate innovati<strong>on</strong><br />

programmes: the MBS Business Informati<strong>on</strong> Systems and the BSc in Food Business with historic<br />

l<strong>on</strong>gitudinal data collecti<strong>on</strong>. This qualitative study reflects <strong>on</strong> the experience of two innovati<strong>on</strong><br />

programmes, the level of analysis, that have been running separately, and in isolati<strong>on</strong>, for between 8-<br />

12 years. These two programmes in the ICT and Food Sectors have resulted in the creati<strong>on</strong> of <strong>on</strong><br />

average twenty student business plans per annum. Forty projects were selected for analysis with an<br />

equal split between IS and Food projects and they were selected to give maximum variati<strong>on</strong> (Miles<br />

and Huberman, 1994). The reas<strong>on</strong> why IS and Food enterprise projects were selected was to<br />

compare innovati<strong>on</strong> intermediati<strong>on</strong> across high technology (Informati<strong>on</strong> Systems) and low technology<br />

(The Food sector being generally regarded as low technology) sectors. The data sources used in this<br />

research included: business plans, learning journals, cross case comparis<strong>on</strong>s and interviews with<br />

students. The data was transcribed, organised, stored and analysed using the software package QSR<br />

NVivo (QSR Internati<strong>on</strong>al, 2002). This computer programme aided in the identificati<strong>on</strong> of codes and<br />

the preparati<strong>on</strong> of codebooks in line with best practise (Cohen et al., 2000; Gilchrist, 1992). Iterative<br />

236


Brian O’Flaherty and Joe Bogue<br />

investigati<strong>on</strong> and systematic coding progressed until a comprehensive interpretati<strong>on</strong> was reached<br />

(Miles and Huberman, 1994).<br />

3. Research results<br />

The results presented in this paper are from an analysis of the roles and functi<strong>on</strong>s of student<br />

enterprise projects over a 5-year project period across two domains: Informati<strong>on</strong> Systems and Food.<br />

The enterprise projects generate <strong>on</strong> average approximately twenty projects per year combined and<br />

vary in terms of the novelty of the technologies used by the student enterprise teams and the<br />

commercial potential of the products/services developed by the student entrepreneurs.<br />

The Business groups engaged in student enterprise in both programmes were drawn randomly at the<br />

beginning of the year, so that students, who were friends, were not all in the same group. This meant<br />

that the groups tended to have students of mixed abilities and also students from different disciplines<br />

as Food Science and Technology students, in the case of the food programme, were also linked into<br />

the groups. This innovative part of the process simulated the linking of marketing and technical<br />

aspects of new product development. The objective of the projects is to develop a technical blueprint<br />

and commercialisati<strong>on</strong> strategy for a new idea or the upgrading of an existing product whilst the<br />

students learn about the process of product innovati<strong>on</strong>.<br />

The scope of the food and software projects varied from those using new technologies (those using<br />

University IP such as in the software, dairy or brewing sectors) to those developing improved versi<strong>on</strong>s<br />

of familiar products (c<strong>on</strong>tinuous innovati<strong>on</strong> in terms of new ingredients, new uses or new packaging).<br />

The Food Business and IS students go through the various stages of new venture creati<strong>on</strong> from<br />

ideati<strong>on</strong> to product launch. An analysis of the data revealed that the students as innovati<strong>on</strong><br />

intermediaries played an important role linking the idea with developing a technical product<br />

specificati<strong>on</strong> and ultimately identifying routes of the product to market. The students across the<br />

different projects played different roles and performed different functi<strong>on</strong>s. Although the Food Business<br />

students were exposed to the entrepreneurial process there were no actual entrepreneurship modules<br />

available to the students. However, they did receive support in relati<strong>on</strong> to the innovati<strong>on</strong> process and<br />

were obliged to make presentati<strong>on</strong>s to staff to highlight key learning outcomes as they progressed<br />

through the innovati<strong>on</strong> funnel.<br />

3.1 Intermediary sources and roles<br />

For this research an analysis was c<strong>on</strong>ducted of the sources of ideas across the forty projects from<br />

both the IS and Food domains. Across both domains, IS and Food, c<strong>on</strong>temporary issues were the<br />

most significant source of project ideas (see table 2) while market research was seen as a more<br />

useful source of ideas in the food domain. The academic project supervisors liaised closely with other<br />

research groups across the University. There were intermediaries in their own right in terms of the<br />

innovati<strong>on</strong> process. Across the Food domain there were multiple ideas generated from University<br />

sources/IP which was indicative of the close link between the Food Business project supervisor and<br />

Food Science research groups. The ideas were identified from the research groups and postdoctoral<br />

researchers also played an important role in project supervisi<strong>on</strong>. The IS groups also relied <strong>on</strong> their<br />

project supervisors for liais<strong>on</strong> with cross university research teams. The research data revealed that<br />

the IS liais<strong>on</strong>s with research groups were less c<strong>on</strong>sistent and more diversified than those in the Food<br />

domain. Where the ideas were sourced from University sources/IP then academic project supervisors<br />

(from IS and Food) played an important gatekeeping/liaising role during the innovati<strong>on</strong> process.<br />

Table 2: Sources of innovati<strong>on</strong> in student enterprise<br />

Domain<br />

Sources of Innovati<strong>on</strong>/ideas<br />

University sources/IP<br />

External company IP<br />

Perceived industry need<br />

C<strong>on</strong>temporary issues<br />

Novelty<br />

Market research<br />

External firms<br />

Informati<strong>on</strong> Systems Food<br />

Legend: No evidence C<strong>on</strong>siderable evidence<br />

237


Brian O’Flaherty and Joe Bogue<br />

The intermediati<strong>on</strong> role of the Food Business students depended to a great extent <strong>on</strong> the<br />

project/product idea and where it was sourced from by the students. In some instances where the<br />

technology/IP was provided to the students by the University or University research groups then the<br />

role they played was to apply the technology to a new situati<strong>on</strong> and to identify a route to market for<br />

that technology. The route to market identified by the students in this instance was heavily influenced<br />

by the supervisory staff within the Univerity. However, where the students themselves identified a<br />

market need, or identified a novel usage, then the students had a str<strong>on</strong>ger influence or sense of<br />

ownership <strong>on</strong> the route to market and also the commercialisati<strong>on</strong> strategy. Where students worked<br />

directly with Food Firms then the Food Firm had a str<strong>on</strong>g influence <strong>on</strong> the outcomes of the innovati<strong>on</strong><br />

process. Where the students worked directly with firms then the students felt that their independence<br />

in the project and its directi<strong>on</strong> was somewhat restricted. In each case the learning outcomes for the<br />

students varied depending <strong>on</strong> the sources of ideas for the enterprise projects.<br />

3.2 Outputs of the innovati<strong>on</strong> projects<br />

Table 3 outlines the output of the student enterprise projects from a sample of forty projects. In the<br />

Food sector there were no patents or licenses developed from the projects. However, <strong>on</strong> the IS side<br />

there was limited evidence of patents and licenses as outputs and across both domains there was<br />

evidence of students becoming entrepreneurs as a result of their enterprise experience. The key<br />

outcome was business plans or a commercial propositi<strong>on</strong>, which c<strong>on</strong>sidered the market and resource<br />

requirement in exploiting a technology.<br />

Table 3: Outputs of student enterprise<br />

Domain<br />

Informati<strong>on</strong> Systems Food<br />

Innovati<strong>on</strong> Outputs<br />

Spinouts<br />

Patents<br />

Licenses<br />

Enterprise grants<br />

Business plans<br />

Student employment<br />

<strong>Academic</strong> papers<br />

Winning enterprise competiti<strong>on</strong>s<br />

Student entrepreneurs<br />

Legend: No evidence C<strong>on</strong>siderable evidence<br />

3.3 Intermediary roles<br />

The analysis of the student enterprise projects identified the source of innovati<strong>on</strong> or idea, rather than<br />

the outcomes, as a defining indicator of the intermediary roles. Likewise the sources are quite broad<br />

and well defined, but the outcomes c<strong>on</strong>sist of predominately business plans with a scattered range of<br />

outliers. The analysis showed that a typology (Tietze, 2010) of three clear intermediary roles emerged<br />

across both the IS and Food entrepreneurship programmes, which were closely aligned to the<br />

sources of innovati<strong>on</strong> (Figures 1, 2, and 3). The student enterprise intermediary roles were: 1)<br />

External scanning intermediaries, 2) University/Technology IP Liais<strong>on</strong> intermediaries and 3) Creative<br />

scanning intermediaries. This Student Enterprise typology is defined as network topologies in line<br />

with Liss<strong>on</strong>i (2008).<br />

The main source of innovati<strong>on</strong> in student enterprise as External Scanning Intermediaries (Figure 1) is<br />

c<strong>on</strong>temporary or popular issues, normally centred <strong>on</strong> a technology or social issue that was receiving<br />

publicity at the time. Across both samples, student teams c<strong>on</strong>verged <strong>on</strong> topical technologies, such as<br />

social media, radio frequency identificati<strong>on</strong> (RFID), probiotic ingredients or ingredients for gluten-free<br />

foods targeted at coeliacs. The student teams exhibited a herding or groupthink mentality, which led<br />

to predicable outcomes. In the food-oriented projects, market research and c<strong>on</strong>sumer behaviour<br />

testing was more comm<strong>on</strong> place, and led to str<strong>on</strong>ger hidden need or evidence based commercial<br />

propositi<strong>on</strong>. While the IS projects tended to retrospectively use market research evidence to justify<br />

initial choices, rather than shaping them. Both analysed segments had examples of innovati<strong>on</strong><br />

sources from industry needs and/or external firms. In some case these could be developed by the<br />

student teams themselves or through industry relati<strong>on</strong>ships with the academic mentors. Examples of<br />

student teams as External Scanning Intermediaries projects were food industry domain projects<br />

238


Brian O’Flaherty and Joe Bogue<br />

targeted at the health and wellness sector and at the super premium indulgent market, while Novel IS<br />

projects developed alternative uses for RFID technology and derivatives trading software that was<br />

d<strong>on</strong>ated by respective manufacturers.<br />

Figure 1: External scanning intermediaries<br />

An interesting and c<strong>on</strong>sistent intermediary role, evident in both the IS and Food programmes, was<br />

Student Teams as University/ technology IP liais<strong>on</strong> intermediaries (Figure 2). In the case of IS<br />

projects, similar technology was given to different teams, which provided numerous commercial<br />

propositi<strong>on</strong>s or product and service ideas that were underpinned by the same technology. These<br />

commercial propositi<strong>on</strong> outcomes could occur in <strong>on</strong>e academic year or over a number of years.<br />

Again the academic mentors played a significant role in sourcing, brokering, gate keeping (Liss<strong>on</strong>i,<br />

2008) the relati<strong>on</strong>ship with the research team or other academic that provided access to the<br />

University Intellectual Property.<br />

Figure 2: University technology/IP liais<strong>on</strong> intermediary<br />

Examples of student teams as University Technology/IP Liais<strong>on</strong> Intermediaries were teams in the<br />

Food sector that exploited IP in probiotics or the broad functi<strong>on</strong>al foods area. IS student teams<br />

applied wireless sensor network (WSN) technology, developed in the Tyndall Institute UCC, <strong>on</strong> a wide<br />

239


Brian O’Flaherty and Joe Bogue<br />

variety of medical and energy applicati<strong>on</strong>s, such as geriatric remote living support, wireless vital signs<br />

m<strong>on</strong>itoring in emergency settings and building energy performance m<strong>on</strong>itoring to the retro-fit home<br />

market.<br />

The curriculum for both academic programmes included significant creativity and opportunity<br />

recogniti<strong>on</strong> and both programme directors strived and encouraged the student teams to c<strong>on</strong>sider<br />

fresh and novel project ideas. Despite this effort, systematising creativity or ‘just being creative’ can<br />

be a difficult outcome. This is where the cultivati<strong>on</strong> of the entrepreneurial mind-set is really evident<br />

and this intermediary role is the most rewarding, but difficult to achieve. Examples of student teams<br />

as Creative Scanning Intermediaries (Figure 3) were found in both sectors. For the food students<br />

many of the students identified their ideas from being <strong>on</strong> internships within food firms or from<br />

spending time abroad during their IS placement programme. Examples of these ideas were the<br />

adapti<strong>on</strong> of a flavour unique to <strong>on</strong>e part of the world (Caribbean) to a new situati<strong>on</strong> (the Irish market)<br />

or the development of a product for a new market (bubble tea from china adapted to the Irish market).<br />

An example of a creative ICT project was a smart ph<strong>on</strong>e applicati<strong>on</strong> for car diagnostics or the use of<br />

gesture technology for interacti<strong>on</strong> with advertising.<br />

Figure 3: Creative scanning intermediary<br />

3.4 Ownership of project IP<br />

A major issue for both sets of students was the ownership of IP when the student enterprise project<br />

was completed. For Food Business students this was not always clear, particularly when they were<br />

working with an external firm or University IP. The questi<strong>on</strong> arose many times: where does ownership<br />

of IP, or what percentage of IP, rests with the students as they have identified route to market for the<br />

IP and also developed a commercialisati<strong>on</strong> strategy. For example, a student enterprise group worked<br />

with an external firm who funded the project and aimed to exploit Univerity IP generated by research<br />

staff. The Food Science students then further developed the technology while the Food Business<br />

students developed the commercialisati<strong>on</strong> plan including the development of a marketing strategy,<br />

brand, packaging and putting together the market research case. In this scenario, the students felt<br />

that they had developed a significant part of the project, from a technical and marketing perspective,<br />

and would need to be compensated in some way if the external firm decided to commercialise the<br />

idea.<br />

3.5 Student enterprise final event<br />

An innovati<strong>on</strong> showcase was held for both IS and Food Business students to introduce the final<br />

project outcomes to industry. This provided the students with the opportunity to display their<br />

endeavours to industry pers<strong>on</strong>nel, academic researchers and those who may be able to<br />

commercialise the research outcomes. Both programmes organised separate Innovati<strong>on</strong> Showcases<br />

as an annual exhibiti<strong>on</strong> of new food and software products developed by the respective students as<br />

part of their final-year-research projects. It was attended by many visitors from industry and other<br />

invited guests, as well as staff and students of the University, who view the selecti<strong>on</strong> of products. The<br />

240


Brian O’Flaherty and Joe Bogue<br />

Showcases allowed the students to highlight their entrepreneurial talents to an assembled audience<br />

and also allowed them to c<strong>on</strong>duct market research by getting c<strong>on</strong>sumers to try samples or just give<br />

feedback <strong>on</strong> the product technology.<br />

The students from both programmes (IS and Food Business) found the showcase to be a great way<br />

to show off their work and also to interact with industry and Technology Transfer Office pers<strong>on</strong>nel. In<br />

additi<strong>on</strong>, where the general public were invited to such exhibiti<strong>on</strong>s, particularly in the food area, it was<br />

seen by students as a first trial run of the product/service and also gave the students a good idea of<br />

how the product would be received in the marketplace, albeit from limited feedback.<br />

4. Research c<strong>on</strong>clusi<strong>on</strong>s<br />

This exploratory study revealed that the university programmes had played an important role in<br />

innovati<strong>on</strong> intermediati<strong>on</strong> and had fostered a culture of innovati<strong>on</strong> within the students. In additi<strong>on</strong>, the<br />

innovati<strong>on</strong> c<strong>on</strong>tributed to a number of business start-ups, development of IP and patents. The original<br />

research questi<strong>on</strong>s focused <strong>on</strong> student teams or enterprise as 1) intermediary roles and 2) the<br />

intermediati<strong>on</strong> process. Students’ endeavour as intermediaries has also added value to local<br />

companies by enhancing existing, or developing, new products or services.<br />

Innovati<strong>on</strong> intermediati<strong>on</strong> underpinned this research, which lead to the identificati<strong>on</strong> of three student<br />

enterprise intermediary roles. The proposed intermediary roles were: 1) External scanning<br />

intermediaries, 2) University/Technology IP Liais<strong>on</strong> intermediaries and 3) Creative scanning<br />

intermediaries. Surprisingly, these roles existed in two distinct and independent entrepreneurship<br />

programmes, which could be categorised as high-tech versus low-tech. The defined roles were<br />

comm<strong>on</strong> across both programmes with little dissimilarity. These subtle differences can be attributed to<br />

different emphasis in curriculum, such as c<strong>on</strong>sumer behaviour in the Food area leading to more<br />

needs driven commercial propositi<strong>on</strong>s.<br />

The outcomes of this study will provide guidelines for other academics, who are currently active or<br />

c<strong>on</strong>sidering developing entrepreneurship courses. The two programmes evolved over nearly a<br />

decade in tandem, but independently, so these findings will allow other academics fast track their<br />

endeavours. The key focus should therefore be <strong>on</strong> cultivating diverse external sources of innovati<strong>on</strong>,<br />

by cultivating relati<strong>on</strong>ships with external companies and mentors, and also encouraging the student<br />

teams to do likewise. The brokering role of the academics in managing the relati<strong>on</strong>ship with<br />

colleagues across the University and adding value to research outcomes, is a surprising outcome,<br />

which is c<strong>on</strong>sistent across both programmes examined. This can also provide a rich source of<br />

innovati<strong>on</strong>, and an untapped resource that can explore commercial outcomes of academic research.<br />

It is known that few scientific academics have the skill set to c<strong>on</strong>sider and develop business<br />

propositi<strong>on</strong>s. The main risk of this type of role relates to the ownership of IP and the extent to which<br />

the c<strong>on</strong>tributi<strong>on</strong> of the intermediating team is recognised. The final role identified centres <strong>on</strong><br />

encouraging student teams to be creative and ‘think outside the box’. Despite the efforts of the<br />

academic in encouraging this, it is difficult to c<strong>on</strong>sistently achieving novel and creative projects.<br />

Future research is also required to better understand the student experience relative to the different<br />

intermediary roles and future cross case analysis will validate and possibly expand <strong>on</strong> the<br />

understanding of these intermediary roles. The intermediati<strong>on</strong> process applied to student enterprise<br />

also warrants further explorati<strong>on</strong>.<br />

References<br />

Batterink, M. H., Wubben, E. F. M., Klerkx, L. and Omta, S. W. F. O. (2010). Orchestrating Innovati<strong>on</strong> Networks:<br />

The Case of Innovati<strong>on</strong> Brokers in the Agri-food Sector. Entrepreneurship Regi<strong>on</strong>al Development, Vol. 22,<br />

No. 1, pp 47-76.<br />

Cohen, L., Mani<strong>on</strong>, L. and Morris<strong>on</strong>, K. (2000). Research Methods in Educati<strong>on</strong>, 5 th Ed., L<strong>on</strong>d<strong>on</strong>,<br />

Routledge/Farmer.<br />

Co<strong>on</strong>ey T. and Murray T. (2008). Entrepreneurship Educati<strong>on</strong> in the Third-level Sector in Ireland, The Nati<strong>on</strong>al<br />

Council for Graduate Entrepreneurship (U.K.).<br />

Dalziel, M. and Parjanen, S. (2012). Measuring the Impact of Innovati<strong>on</strong> Intermediaries: A Case Study of Tekes.<br />

In Melkas, H. and Harmaakorpi, V. (eds.) Practice-based innovati<strong>on</strong>: Insights, applicati<strong>on</strong>s and policy<br />

implicati<strong>on</strong>s, Springer.<br />

Fitzsim<strong>on</strong>s, P. and O’ Gorman, C. (2005). The Global Entrepreneurship M<strong>on</strong>itor; The Irish Report, Dublin: The<br />

Global Entrepreneurship M<strong>on</strong>itor.<br />

Gassmann, O., Daiber, M. and Enkel, E. (2011). The Role of Intermediaries in Cross-industry Innovati<strong>on</strong><br />

Processes, R&D <strong>Management</strong>, Vol. 41, No. 5, pp 457–469.<br />

241


Brian O’Flaherty and Joe Bogue<br />

Gilchrist, V.J. (1992). “Sampling in Qualitative Inquiry”. In: Doing Qualitative Research: Research Methods for<br />

Primary Care Vol. 3 (Crabtree, B.F. and Miller, W.L., Eds.), pp 31-44, Newbury Park, California, Sage.<br />

Goodbody Ec<strong>on</strong>omic C<strong>on</strong>sultants. (2002). Entrepreneurship in Ireland, November Dublin: Goodbody Ec<strong>on</strong>omic<br />

C<strong>on</strong>sultants.<br />

Gord<strong>on</strong>, I., Jack, S. L. and Hamilt<strong>on</strong>, E. E. (2010). A Study of the Regi<strong>on</strong>al Ec<strong>on</strong>omic Development Impact of a<br />

University-led Entrepreneurship Educati<strong>on</strong> Programme for Small Business Owners, Entrepreneurship and<br />

Enterprise Development Working Paper Series, Lancaster University <strong>Management</strong> School Working Paper<br />

series No. 2010/02.<br />

Hoppe, H. C. and E. Ozdenoren. (2005). "Intermediati<strong>on</strong> in Innovati<strong>on</strong>." Internati<strong>on</strong>al Journal of Industrial<br />

Organizati<strong>on</strong>, Vol. 23, No. 5–6, pp 483-503.<br />

Hisrich, R.D., Peters, M.P. and Shepherd, D.A. (2005). Entrepreneurship, 6 th Editi<strong>on</strong>, New York, McGraw-Hill.<br />

Howells, J. (2006). "Intermediati<strong>on</strong> and the Role of Intermediaries in Innovati<strong>on</strong>." Research Policy, Vol. 35, No.<br />

5, pp 715-728.<br />

Klerkx, L., and Leeuwis, C. (2008). “Balancing Multiple Interests: Embedding Innovati<strong>on</strong> Intermediati<strong>on</strong> in the<br />

Agricultural Knowledge Infrastructure”. Technovati<strong>on</strong>, Vol. 28, No. 6, pp 364-378.<br />

Lichtenthaler, U. and Ernst, H. (2008). “Innovati<strong>on</strong> Intermediaries: why Internet Marketplaces for Technology<br />

have not yet met the Expectati<strong>on</strong>s”. Creativity and Innovati<strong>on</strong> <strong>Management</strong>, Vol. 17, No. 1, pp 14-25.<br />

Liss<strong>on</strong>i, F. (2008). "<strong>Academic</strong> Inventors as Brokers: An Exploratory Analysis of the KEINS Database," KITeS<br />

Working Papers 213, KITeS, Centre for Knowledge, Internati<strong>on</strong>alizati<strong>on</strong> and Technology Studies, Universita<br />

Bocc<strong>on</strong>i, Milano, Italy.<br />

Metcalfe J. S. (2009). University and Business Relati<strong>on</strong>s: C<strong>on</strong>necting the Knowledge Ec<strong>on</strong>omy, Centre for<br />

Business Research, University of Cambridge Working Paper No. 395, December 2009.<br />

Miles, M.B. and Huberman, A.M. (1994). Qualitative Data Analysis 2 nd Ed., L<strong>on</strong>d<strong>on</strong>, Sage.<br />

O'Gorman, C. and P. Fitzsim<strong>on</strong>s. (2007). Entrepreneurial Activity in Ireland: Evidence from the Global<br />

Entrepreneurship M<strong>on</strong>itor. Irish Journal of <strong>Management</strong>, Vol. 28, No. 2.<br />

QSR Internati<strong>on</strong>al. (2002). N6 (n<strong>on</strong>-numerical Unstructured Data Indexing Searching and Theorizing) Qualitative<br />

Data Analysis Program. Australia: QSR Internati<strong>on</strong>al Pty Ltd. Versi<strong>on</strong> 6.0.<br />

Saari S. and Haapasalo H. (2012).Knowledge Transfer Processes in Product Development —Theoretical<br />

Analysis in Small Technology Parks, Technology and Investment, Vol. 3, pp 36-47.<br />

SSTI. (2006). Strategy for Science, Technology and Innovati<strong>on</strong> 2006-2013, DETE, Ireland, June.<br />

Streeter D. H. and Jaquette, J. P. (2004). “University-wide Entrepreneurship Educati<strong>on</strong>: Alternative Models and<br />

Current Trends”, Southern Rural Sociology, Vol. 20, No. 2, pp 44 - 71.<br />

Tietze, F. (2010). A Typology of Technology Market Intermediaries Technology and Innovati<strong>on</strong>s <strong>Management</strong><br />

Working Paper No. 60.<br />

World Ec<strong>on</strong>omic Forum. (2009). World Ec<strong>on</strong>omic Forum Report <strong>on</strong> Entrepreneurship Educati<strong>on</strong>, ‘Educating the<br />

Next Wave of Entrepreneurs - Unlocking entrepreneurial capabilities to meet the global challenges of the<br />

21st Century’, April 2009.<br />

Yusuf, S. (2008). "Intermediating Knowledge Exchange Between Universities and Businesses," Research Policy,<br />

September, Vol. 37, No. 8, pp 1167-1174.<br />

242


Bringing Some Order to the ‘Black Art’ of Innovati<strong>on</strong><br />

Measurement<br />

Paidi O’Raghallaigh, David Samm<strong>on</strong> and Ciaran Murphy<br />

Business Informati<strong>on</strong> Systems, University College Cork, Cork, Ireland<br />

paidioreilly@gmail.com<br />

dsamm<strong>on</strong>@afis.ucc.ie<br />

cmurphy@afis.ucc.ie<br />

Abstract: Measurement of innovati<strong>on</strong> is critical to management but unfortunately it is an extremely tall order,<br />

which results in it being referred to as a ‘black art’. It is particularly troublesome for firms, which operate in highly<br />

complex and turbulent envir<strong>on</strong>ments. Extant literature is characterized by a diversity of approaches,<br />

prescripti<strong>on</strong>s, and practices that are more often than not c<strong>on</strong>fusing and c<strong>on</strong>tradictory. We require good theory<br />

both to suggest which metrics are needed and to interpret the resulting data. In this paper we return to the<br />

literature in order to build a c<strong>on</strong>ceptual framework to guide the measurement of innovati<strong>on</strong>. In additi<strong>on</strong> we<br />

perform an initial validati<strong>on</strong> of the framework against an <strong>on</strong>line repository of c<strong>on</strong>tent <strong>on</strong> innovati<strong>on</strong> measurement<br />

and in so doing we arrive at a tax<strong>on</strong>omy of innovati<strong>on</strong> metrics. While useful in its own right, the tax<strong>on</strong>omy also<br />

highlights both the strengths and weaknesses in the current approach to innovati<strong>on</strong> measurement. Finally we use<br />

the framework to draw out four key questi<strong>on</strong>s that should be addressed by management before choosing<br />

appropriate metrics. We foresee management using both the tax<strong>on</strong>omy and the guiding questi<strong>on</strong>s to evaluate<br />

their own measurement activities.<br />

Keywords: innovati<strong>on</strong>, innovati<strong>on</strong> measurement, innovati<strong>on</strong> metrics, innovati<strong>on</strong> model<br />

1. Introducti<strong>on</strong><br />

Measurement of innovati<strong>on</strong> is critical for both practiti<strong>on</strong>ers and academics and is where the rubber<br />

meets the road. It makes organizati<strong>on</strong>s aware of where they are in relati<strong>on</strong> to their goals, in which<br />

directi<strong>on</strong> they are travelling, and any corrective acti<strong>on</strong>s they may need. But for most organisati<strong>on</strong>s<br />

innovati<strong>on</strong> measurement is an extremely tall order, with some going as far as describing it as a ‘black<br />

art’. Even for those that do measure innovati<strong>on</strong>, it is comm<strong>on</strong> for doubts to persist as to whether they<br />

are using appropriate metrics and for the right reas<strong>on</strong>s. Unfortunately, the extant literature <strong>on</strong><br />

innovati<strong>on</strong> measurement is flung far and wide and in any case is characterized by a diversity of<br />

approaches, prescripti<strong>on</strong>s, and practices that are more often than not c<strong>on</strong>fusing and c<strong>on</strong>tradictory<br />

(Adams et al. 2006). Innovati<strong>on</strong> measurement has always been a thorny issue for researchers<br />

(Archibugi and Planta 1996; Becheikh et al. 2006; Hagedoorn and Cloodt 2003). But the focus of our<br />

attenti<strong>on</strong> here is <strong>on</strong> the measurement of innovati<strong>on</strong> within management practice, which has received<br />

little scholastic attenti<strong>on</strong> (Kerssens van Dr<strong>on</strong>gelen and Bilderbeek 1999). We attempt to address this<br />

shortcoming by addressing the research questi<strong>on</strong> of how organisati<strong>on</strong>s should choose appropriate<br />

innovati<strong>on</strong> metrics. While it may be easy to define metrics, it is more difficult to identify meaningful<br />

<strong>on</strong>es (Böhme and Freiling 2008).<br />

Before we begin our journey it is important that we define the c<strong>on</strong>cepts of measurements, metrics and<br />

models - an understanding of which is fundamental to the topic of this paper. A measurement is the<br />

use of an appropriate method to collect data from an observati<strong>on</strong> of an object of interest. To measure<br />

means to attach a number or category to an attribute that represents some aspect of the object<br />

(Böhme and Freiling 2008). Metrics are the rules that assign the resulting data <strong>on</strong>to a scale in order to<br />

appropriately represent the attribute of interest. For example, the temperature of a room can be<br />

observed from a thermometer, which codes the observed mercury level using a metric, such as<br />

degrees Fahrenheit. But the metric does not necessarily need to comprise of numbers but could<br />

equally c<strong>on</strong>sist of categories. For example, the temperature in the room could be categorised as “hot”,<br />

“warm”, or “cool”. A model is a formal representati<strong>on</strong> of the object of interest in terms of c<strong>on</strong>cepts that<br />

are necessary and sufficient to describe the object. Models are used in informing the choice of metrics<br />

and are particularly required when there are n<strong>on</strong>trivial relati<strong>on</strong>ships between possible measurements<br />

and the attributes..<br />

The remainder of this paper is structured as follows. We begin by outlining the importance of<br />

measurement to management. We follow this by suggesting a framework to assist management in<br />

choosing metrics. Using the framework we next analyse the state of the art of innovati<strong>on</strong><br />

measurement. We finish by offering guidelines to assist management in measuring innovati<strong>on</strong>.<br />

243


Paidi O’Raghallaigh, David Samm<strong>on</strong> and Ciaran Murphy<br />

2. Importance of measurement to management<br />

According to Fayol the functi<strong>on</strong>s of management include: forecasting and planning, organizing,<br />

commanding, coordinating, and c<strong>on</strong>trolling (Carroll and Gillen 1987). We take managerial c<strong>on</strong>trol,<br />

which Mockler (1970 p. 14) defines as “a systematic effort by business management to compare<br />

performance to predetermined standards, plans, or objectives in order to determine whether<br />

performance is in line with these standards and presumably in order to take any remedial acti<strong>on</strong><br />

required to see that human and other corporate resources are being used in the most effective and<br />

efficient way possible in achieving corporate objectives” as our starting point. This ‘cycle of c<strong>on</strong>trol’<br />

includes the strategic planning, the measuring activities to determine whether targets are being<br />

achieved, and the correcting or changing of activities as needed (Daft 1998).<br />

Traditi<strong>on</strong>ally c<strong>on</strong>trol systems emphasise ‘executi<strong>on</strong> rather than explorati<strong>on</strong>’ (Davila et al. 2009). But<br />

innovati<strong>on</strong> efforts prosper in envir<strong>on</strong>ments that promote experimentati<strong>on</strong> (Amabile 1999).<br />

<strong>Management</strong> c<strong>on</strong>trol, through imposing c<strong>on</strong>straints <strong>on</strong> behaviour, would, therefore, be expected to<br />

reduce the creativity that may be necessary when innovating (Amabile 1999; Davila et al. 2009; Ditillo<br />

2004). But in recent years, changing business envir<strong>on</strong>ments are challenging organisati<strong>on</strong>s to improve<br />

the management of their innovati<strong>on</strong> efforts (Davila et al. 2009). “More prosaically, this means<br />

organizati<strong>on</strong>s need to be able, for example, to provide sufficient freedom to allow for the explorati<strong>on</strong> of<br />

creative possibilities, but sufficient c<strong>on</strong>trol to manage innovati<strong>on</strong> in an effective and efficient fashi<strong>on</strong>”<br />

(Adams et al. 2006 p. 32).<br />

But when managed poorly, measurement can be a double-edged sword. While metrics can drive<br />

positive change throughout an organizati<strong>on</strong>, they can also generate unintended negative<br />

c<strong>on</strong>sequences that hinder organisati<strong>on</strong>al performance (Collins and Smith 1999). Dysfuncti<strong>on</strong>al<br />

behaviour can arise from the design and use of inappropriate measurements (Jaworski 1988). When<br />

faced by inappropriate measurements, managers may cope by resisting or circumventing c<strong>on</strong>trols by<br />

behaving in a fashi<strong>on</strong> that is in his or her (rather than the organisati<strong>on</strong>’s) best interest (Jaworski<br />

1988). So how should we ensure that measurements are chosen to make a positive impact <strong>on</strong> the<br />

process of innovati<strong>on</strong>?<br />

3. Framework for choosing innovati<strong>on</strong> metrics<br />

Innovati<strong>on</strong> management is somewhat of a ‘black art’ and managers currently lack the requisite metrics<br />

to make informed decisi<strong>on</strong>s about their innovati<strong>on</strong> programs (Muller et al. 2005). Indeed innovati<strong>on</strong><br />

does not lend itself to measurement. Shapiro (2006 p. 42) states that “The essence of innovati<strong>on</strong> is<br />

novelty … It may even be that the most effective innovati<strong>on</strong> is that which so changes the scheme of<br />

things that it makes the old measuring scheme obsolete!” This does not mean that measurement<br />

approaches cannot be developed, but such approaches require us to reduce the innovati<strong>on</strong><br />

phenomen<strong>on</strong> to some simplified and stable c<strong>on</strong>ceptualisati<strong>on</strong> (Smith 1998). An accepted c<strong>on</strong>ceptual<br />

framework is, therefore, necessary to guide measurement activities and to move innovati<strong>on</strong><br />

management bey<strong>on</strong>d an art to a scientific basis. Osterwalder et al., 2005 (2005 p. 21) suggests that a<br />

framework makes it “easier to identify the relevant measures to follow to improve management. This<br />

ability would facilitate the choice of the indicators … for m<strong>on</strong>itoring strategy implementati<strong>on</strong>”.<br />

But despite decades of academic attenti<strong>on</strong> the innovati<strong>on</strong> process in organisati<strong>on</strong>s remains<br />

imperfectly understood by both academia and management (Becheikh et al. 2006). But we require<br />

“good theory to both suggest which indicators are needed and to interpret the resulting data, and an<br />

effective … resp<strong>on</strong>se to identified problems” (Arundel et al. 1998 p. 4). We, therefore, find ourselves<br />

returning to the literature in order to build a c<strong>on</strong>ceptual framework of the innovati<strong>on</strong> process. We draw<br />

<strong>on</strong> the management c<strong>on</strong>trol literature, briefly introduced above, to identify three categories of<br />

measurements - each distinguished by the timing of management interventi<strong>on</strong> (i.e., input to output) in<br />

the innovati<strong>on</strong> process. Resource c<strong>on</strong>trols are measurable acti<strong>on</strong>s taken by management usually<br />

prior to implementati<strong>on</strong> of a process in order to allocate resources (Jaworski 1988). Process c<strong>on</strong>trols<br />

are measurable acti<strong>on</strong>s taken by management usually during the process in order to influence the<br />

behaviour of actors and activities involved in the process (ibid). Result c<strong>on</strong>trols are measurable<br />

acti<strong>on</strong>s taken by management usually after executi<strong>on</strong> of the process in setting, m<strong>on</strong>itoring, and<br />

evaluating performance standards (ibid). Using this classificati<strong>on</strong> we now develop a framework of the<br />

innovati<strong>on</strong> process.<br />

244


3.1 The process of innovati<strong>on</strong><br />

Paidi O’Raghallaigh, David Samm<strong>on</strong> and Ciaran Murphy<br />

Various scholars (e.g. Hansen and Birkinshaw 2007; Kandybin and Kihn 2004; Koen et al. 2001),<br />

albeit sometimes using slightly different terminology, view the innovati<strong>on</strong> process as c<strong>on</strong>sisting of the<br />

generati<strong>on</strong>, selecti<strong>on</strong>, development, and commercialisati<strong>on</strong> of knowledge – see Table 1. We d<strong>on</strong>’t<br />

intend to imply a well-behaved single, orderly, and linear process but instead the process, especially<br />

in its early stages, is more often than not chaotic, unpredictable, and unstructured before in its later<br />

stages becoming more structured (Koen 2004). This journey is often marked by dead-ends, re-births,<br />

and reversals as the knowledge is rejected, re-introduced, or reworked (Cagan and Vogel 2002).<br />

While there is c<strong>on</strong>siderable value in viewing innovati<strong>on</strong> as a staged process it hides the variety and<br />

complexity of the individual activities that occur both within and across stages. We now take a look at<br />

the characteristics of these activities.<br />

Table 1: Stages of knowledge progressi<strong>on</strong> (after: Hansen and Birkinshaw 2007; Kandybin and Kihn<br />

2004)<br />

Generati<strong>on</strong> Selecti<strong>on</strong> Development Commercialisati<strong>on</strong><br />

The process starts with<br />

the generati<strong>on</strong> of good<br />

ideas. Viable ideas are<br />

usually ignited when<br />

fragments of knowledge<br />

come together from<br />

different sources – both<br />

internal and external.<br />

Idea generati<strong>on</strong> may be<br />

either market-pull<br />

(through market needs)<br />

or technology-push<br />

(through technology<br />

advances).<br />

The pool of ideas<br />

generated in the<br />

previous stage is<br />

funneled into a smaller<br />

number of funded<br />

projects. Not <strong>on</strong>ly is it<br />

possible to miss some<br />

good ideas but<br />

accepting too many<br />

ideas is also an issue<br />

resulting in too many<br />

‘bad’ ideas being funded<br />

and resources being<br />

wasted <strong>on</strong> projects that<br />

never reach market.<br />

The funded projects are<br />

developed into revenuegenerating<br />

deliverables.<br />

Expediting development<br />

lowers the costs (both<br />

the direct costs of<br />

producing the<br />

deliverable but also the<br />

opportunity costs of not<br />

being able to produce<br />

other deliverables).<br />

Attenti<strong>on</strong> now begins to<br />

switch from the<br />

innovati<strong>on</strong> value-chain<br />

to the product supplychain<br />

to ensure that the<br />

deliverables are “where<br />

they need to be when<br />

they’re needed”, and to<br />

promote and market<br />

them intelligently. This<br />

depends <strong>on</strong> getting buyin<br />

both internally and<br />

externally.<br />

The challenge for organisati<strong>on</strong>s is to turn knowledge from both internal and external sources into<br />

exploitable knowledge (Kogut and Zander 1996). But the ability of the organisati<strong>on</strong> to do this depends<br />

<strong>on</strong> the proximity of the knowledge sources relative to the existing stock of knowledge in the<br />

organisati<strong>on</strong> (O'Raghallaigh et al. 2010). Two critical dimensi<strong>on</strong>s of proximity are territorial and<br />

cognitive distances, where the former is the geographical distance of the knowledge source to the<br />

focal firm and the latter is the familiarity of the knowledge source to the focal firm (ibid). The<br />

innovati<strong>on</strong> activities of the organisati<strong>on</strong> are essentially knowledge search activities that can be<br />

distinguished in relati<strong>on</strong> to these two dimensi<strong>on</strong>s. Territorial proximity is important as smaller<br />

distances facilitate more intense interacti<strong>on</strong>s, reduce instituti<strong>on</strong>al and culture differences, and thereby<br />

promote knowledge transfer - especially of sticky tacit knowledge (ibid). On the other hand, cognitive<br />

diversity is important in promoting novelty, which can be critical to innovati<strong>on</strong> performance (ibid).<br />

Search activities al<strong>on</strong>g the territorial dimensi<strong>on</strong> can be divided into internal, dynamic, and market<br />

categories (O'Raghallaigh et al. 2010). Internal search activities focus <strong>on</strong> the acquisiti<strong>on</strong> of knowledge<br />

within the boundaries of the organisati<strong>on</strong>, such as through R&D and daily operati<strong>on</strong>s (ibid). Market<br />

search activities focus <strong>on</strong> acquiring external knowledge through market-driven transacti<strong>on</strong>s, such as<br />

inward-licensing, purchasing of patents, purchasing of equipment, etc. (ibid). The knowledge may be<br />

acquired in either an embodied (e.g. in machines and equipment) or disembodied (e.g. through<br />

licensing agreements or R&D outsourcing c<strong>on</strong>tracts) state (ibid). It is also possible to acquire external<br />

knowledge without engaging with the market. Dynamic activities refer to efforts at acquiring external<br />

knowledge through cooperating with external organisati<strong>on</strong>s (i.e. suppliers, customers, and<br />

universities) and scanning external informati<strong>on</strong> sources (i.e. by attending c<strong>on</strong>ferences, reading<br />

scientific publicati<strong>on</strong>s, and reading technical reports) (ibid). Unlike market activities, which are<br />

essentially arms-length market transacti<strong>on</strong>s, dynamic activities require the active participati<strong>on</strong> of the<br />

to-be recipient in the process of knowledge acquisiti<strong>on</strong> (Cassiman and Veugelers 2002). An<br />

organisati<strong>on</strong> can, therefore, innovate by producing knowledge through its own inventive activities but<br />

also through obtaining it from sources outside of the firm. The latter may involve “no intellectual,<br />

inventive, or creative effort whatsoever” <strong>on</strong> behalf of the focal organisati<strong>on</strong>, while the former does<br />

(Arundel et al. 1998 p. 31). These two aspects of innovati<strong>on</strong> are referred to as innovati<strong>on</strong> through<br />

inventive effort and innovati<strong>on</strong> through adopti<strong>on</strong> (ibid). The ways in which each approach is managed<br />

is likely to differ greatly.<br />

245


Paidi O’Raghallaigh, David Samm<strong>on</strong> and Ciaran Murphy<br />

To foster innovati<strong>on</strong> within organisati<strong>on</strong>s, resources must be distributed deliberately based <strong>on</strong> predefined<br />

goals (Alves et al. 2007). Christensen (2006 p. xvii-xviii) argues that “organizati<strong>on</strong>s<br />

successfully tackle opportunities when they have the resources to succeed, when their processes<br />

facilitate what needs to get d<strong>on</strong>e, and when their values allow them to give adequate priority to that<br />

particular opportunity in the face of all other demands that compete for the company’s resources”.<br />

Innovati<strong>on</strong> resources are widely portrayed as including funding, human, facilities, and tools resources<br />

(Adams et al. 2006) – see Table 2.<br />

Table 2: Types of innovati<strong>on</strong> resources (after: Adams et al. 2006)<br />

Adequate funding is clearly a critical input into the innovati<strong>on</strong> process and funding may need to<br />

Funding<br />

be designated to specific activities.<br />

People factors include the number and mix (with respect to their cosmopolitanism, propensity to<br />

innovate, skills, experience, and educati<strong>on</strong>) of people committed to the innovati<strong>on</strong> tasks.<br />

Human<br />

Members with higher levels of educati<strong>on</strong> and self-esteem from diverse backgrounds increase<br />

Facilities<br />

the effectiveness of innovati<strong>on</strong> project teams.<br />

Facilities or physical resources are a broad category that ranges from buildings to computer<br />

equipment. Slack resources or unused capacity can in some cases be an important catalyst for<br />

innovati<strong>on</strong>, whereby slack provides the opportunity for diversificati<strong>on</strong>, fosters a culture of<br />

experimentati<strong>on</strong>, protects against the uncertainty of project failure, and allows failures to be<br />

absorbed. However, in other cases slack becomes syn<strong>on</strong>ymous with waste and is a cost that<br />

must be eliminated.<br />

Use of systems and tools is an important support for innovati<strong>on</strong> in organizati<strong>on</strong>s. Tools can be of<br />

various sorts, including tools and techniques for promoting creativity and systems of quality<br />

Tools<br />

c<strong>on</strong>trol ranging from informal methods to specific techniques such as total quality management<br />

(TQM).<br />

Innovati<strong>on</strong> can take place <strong>on</strong> any dimensi<strong>on</strong> of the organisati<strong>on</strong>’s business model (Sawhney et al.<br />

2006). Innovati<strong>on</strong> is measured is terms of the value it adds to organisati<strong>on</strong> rather than the novelty of<br />

its results. For instance a new product that is technologically superior to its competitors and meets the<br />

needs of a customer segment can fail because it lacks an effective sales and distributi<strong>on</strong> channel.<br />

When innovating, an organisati<strong>on</strong> must c<strong>on</strong>sider all dimensi<strong>on</strong>s across which innovati<strong>on</strong> can take<br />

place - see Table 3.<br />

Table 3: Types of innovati<strong>on</strong> results (after: Sawhney et al. 2006)<br />

These are goods and services offered by the organisati<strong>on</strong> and that are valued by its<br />

Offerings<br />

customers.<br />

This is a set of comm<strong>on</strong> comp<strong>on</strong>ents, assembly methods, or technologies from the<br />

Platform<br />

organisati<strong>on</strong> that serve as building blocks for a wider portfolio of offerings.<br />

This is the customized, integrated combinati<strong>on</strong> of offerings from the organisati<strong>on</strong> that<br />

Soluti<strong>on</strong>s<br />

solves a customer problem.<br />

This is the discovery by the organisati<strong>on</strong> of new customer segments or the uncovering of<br />

Customers<br />

unmet (and sometime unarticulated) customer needs in existing segments.<br />

Customer This includes everything a customer sees, hears, feels and in general experiences while<br />

Experience<br />

interacting with the organisati<strong>on</strong> and its offerings.<br />

Value This is the mechanism that the organisati<strong>on</strong> uses to capture revenue streams from the<br />

Capture<br />

value it creates.<br />

These are the c<strong>on</strong>figurati<strong>on</strong>s of business activities that the organisati<strong>on</strong> uses to c<strong>on</strong>duct<br />

Processes<br />

internal operati<strong>on</strong>s.<br />

This is the way in which the organisati<strong>on</strong> is structured, its partnerships, and its employee<br />

Organizati<strong>on</strong><br />

roles and resp<strong>on</strong>sibilities.<br />

This is the sequence of agents, activities and resources required by the organisati<strong>on</strong> to<br />

Supply Chain<br />

move its offerings from source to the customer.<br />

Points of These are the channels and the outlets that the organisati<strong>on</strong> employs from which its<br />

Presence<br />

offerings can be bought or used by the customer.<br />

This is the network through which the organisati<strong>on</strong> and its offerings are c<strong>on</strong>nected to the<br />

Networking<br />

customer.<br />

This is the set of symbols, words or marks through which the organisati<strong>on</strong> communicates a<br />

Brand<br />

promise to the customer.<br />

The resulting synthesis of the stages, activities, resources, and results of the innovati<strong>on</strong> process is<br />

depicted in the framework shown in Figure 2.<br />

246


Paidi O’Raghallaigh, David Samm<strong>on</strong> and Ciaran Murphy<br />

Figure 2: C<strong>on</strong>ceptual framework of innovati<strong>on</strong> process<br />

4. Initial evaluati<strong>on</strong> of the ‘state of the art’ in innovati<strong>on</strong> metrics<br />

Year after year, results from surveys by the Bost<strong>on</strong> C<strong>on</strong>sulting Group show how most executives are<br />

aware of the importance of measuring innovati<strong>on</strong> rigorously, c<strong>on</strong>sistently, and effectively but few of<br />

them follow through in terms of executi<strong>on</strong>. In order to ‘sample’ the state of the art in innovati<strong>on</strong><br />

measurement, we reviewed the extensive c<strong>on</strong>tent available in the Innovati<strong>on</strong> Metrics secti<strong>on</strong> of the<br />

Business Exchange 1 , which is a repository c<strong>on</strong>taining links to articles, news, blogs, and general<br />

resources about innovati<strong>on</strong> metrics from myriad top <strong>on</strong>line sources. A group of seven postgraduate<br />

students from a Masters Programme of Innovati<strong>on</strong> Studies initially mined the repository for details of<br />

innovati<strong>on</strong> metrics used by organisati<strong>on</strong>s. A blind review of the same material was simultaneously<br />

performed by the corresp<strong>on</strong>ding author and the results were validated against those derived by the<br />

students. The author overlaid the uncovered metrics <strong>on</strong>to the framework described in the previous<br />

secti<strong>on</strong>. In all 175 sources were reviewed and each of the resulting metrics was successfully<br />

incorporated into the framework – see Figure 3. In the first instance this validated the ability of the<br />

framework to handle all uncovered metrics. However, reflecti<strong>on</strong> <strong>on</strong> the results leads to a number of<br />

interesting observati<strong>on</strong>s, which we discuss briefly here. These observati<strong>on</strong>s are mainly drawn from<br />

the perspective of software-producing firms, for which the measurement problem is particularly acute.<br />

As may have been expected the majority of the uncovered metrics are c<strong>on</strong>centrated in the results<br />

category and to a lesser extent the resources category. When measuring results the emphasis is very<br />

much <strong>on</strong> financial measures of product performance. This somewhat corroborates the finding that<br />

‘Percent of revenue from new products’ is the most comm<strong>on</strong>ly used measure of innovati<strong>on</strong> in<br />

organisati<strong>on</strong>s (Shapiro 2006). While this metric is widely used, it (and similar result-metrics) is not<br />

without its problems. A general problem is the questi<strong>on</strong> of what is ‘new’. Would even the most basic<br />

change (such as colour, size, packaging, raw materials, etc.) to an existing product result in a new<br />

product (Shapiro 2006). Sec<strong>on</strong>dly, it does not reflect the size of the original investment in the<br />

development of the innovati<strong>on</strong> and neither does it indicate the profitability of the investment (Linder<br />

2006). Thirdly, it is backward focused “capturing the impact of past innovati<strong>on</strong>s but … [not] current<br />

investments and whether or not they will pay off in the future” (ibid p. 38). Fourthly, it does not indicate<br />

how the organisati<strong>on</strong> is performing relative to, say, others in the industry (ibid). Fifthly, it does not<br />

reflect the value of those types of innovati<strong>on</strong>s that do not directly result in sales (ibid). While customer<br />

experience metrics were uncovered, all other forms of innovati<strong>on</strong> (e.g. process, organizati<strong>on</strong>al,<br />

supply-chain, etc.) hardly register at all. Finally, the speed to market for software firms is paramount<br />

and market leadership is maintained by actively removing opportunities for competitors to jump into a<br />

market leadership positi<strong>on</strong> (Hoch et al. 2000). It may be necessary for organisati<strong>on</strong>s to "cannibalise"<br />

their own products with the result that maximisati<strong>on</strong> of revenue from existing products may be<br />

1 The service is offered by BusinessWeek a top ranking financial and business magazine (with an in print circulati<strong>on</strong> of circa 1<br />

milli<strong>on</strong> copies) and whose editors vet all links for appropriateness to the topic. See http://bx.businessweek.com/innovati<strong>on</strong>metrics/<br />

247


Paidi O’Raghallaigh, David Samm<strong>on</strong> and Ciaran Murphy<br />

sec<strong>on</strong>dary to insuring and increasing market share (ibid). The uncovered metrics can reflect this<br />

scenario through metrics dealing with rates of product uptake and market share.<br />

Figure 3: Map of innovati<strong>on</strong> metrics<br />

248


Paidi O’Raghallaigh, David Samm<strong>on</strong> and Ciaran Murphy<br />

Our results show that organisati<strong>on</strong>s devote c<strong>on</strong>siderable attenti<strong>on</strong> to measuring funding and human<br />

resources – both being critical inputs into the innovati<strong>on</strong> process. But other resource types, such as<br />

facilities and tools, receive little or no attenti<strong>on</strong>. Yet both are crucial to innovati<strong>on</strong> in software firms. A<br />

disadvantage of focusing too closely or exclusively <strong>on</strong> resources is that they tell us nothing about the<br />

process of c<strong>on</strong>versi<strong>on</strong> by which they are c<strong>on</strong>verted into results. <strong>Management</strong> of knowledge in terms of<br />

the acquisiti<strong>on</strong>, generati<strong>on</strong> and exploitati<strong>on</strong> of knowledge is critical to the success of software firms<br />

(Mathiassen and Vainio 2007). While there is a breath in the process-metrics uncovered, the depth of<br />

the metrics varies from stage to stage – with far more metrics focusing <strong>on</strong> the idea generati<strong>on</strong> stage<br />

than any other stage. Surprisingly, there is little emphasis <strong>on</strong> measuring the role of collaborati<strong>on</strong> in the<br />

innovati<strong>on</strong> process of organizati<strong>on</strong>s. No matter how large, no <strong>on</strong>e software company can achieve<br />

market leadership by itself as gaps in technological, marketing and R&D expertise spring up all the<br />

time (Nambisan 2002). Leading software firms increasingly realize the critical importance of partnering<br />

in filling gaps in expertise, speeding up time to market, increasing market penetrati<strong>on</strong>, and supporting<br />

R&D efforts (Hoch et al. 2000). Our evidence shows that outside of some internal R&D metrics, the<br />

uncovered metrics are not designated at the level of the specific activity-types – such as<br />

collaborati<strong>on</strong>s, scanning, and market acquisiti<strong>on</strong>s.<br />

5. Discussi<strong>on</strong><br />

Case studies of innovati<strong>on</strong> have c<strong>on</strong>sistently dem<strong>on</strong>strated the c<strong>on</strong>siderable complexity and diversity<br />

of innovati<strong>on</strong> processes across firms and industries (Smith 1998), which leads <strong>on</strong>e to expect great<br />

difficulty in the measurement of innovati<strong>on</strong>. When it comes to choosing metrics ‘<strong>on</strong>e size does not fit<br />

all’ and instead the optimal selecti<strong>on</strong> of metrics will vary between organisati<strong>on</strong>s and industries (Muller<br />

et al. 2005). Unfortunately when faced with measurement issues, management generally just add<br />

more metrics (Linder 2006). But measurement is not a matter of using as many metrics as possible -<br />

there is a cost associated with measuring and in any case what is to say that the added metrics solve<br />

the underlying measurement issue. We recommend a more scientific approach to measurement,<br />

which demands an in-depth understanding of the c<strong>on</strong>text in which innovati<strong>on</strong> takes places. Metrics<br />

should not be mistaken for what is being measured and they are no more than proxies for an abstract<br />

reality. No measurement problem can be adequately addressed without answering the questi<strong>on</strong>s in<br />

Table 4 - the answers to which inform both the choice of metrics and the balance of metrics within the<br />

overall portfolio used by the focal organisati<strong>on</strong>. The questi<strong>on</strong>s are derived from the previously<br />

discussed framework and in particular from its four elements – Goals, Resources, Results, and<br />

Process. The metrics chosen must be aligned to these elements in order to ensure their<br />

appropriateness. In additi<strong>on</strong> the resulting metric portfolio must be sufficiently broad to adequately<br />

cover resources, process, and results. Finally, the metrics may need to be designated at the level of<br />

distinct activity categories (e.g. collaborati<strong>on</strong>, scanning, market acquisiti<strong>on</strong>s, etc.). This paper has<br />

sought to make a number of key c<strong>on</strong>tributi<strong>on</strong>s. Most importantly, we acknowledge the difficulties that<br />

management are experiencing when measuring innovati<strong>on</strong>. We expect that this paper will give both<br />

managers and academics firmer grounds <strong>on</strong> which to (1) understand and appraise their measurement<br />

activities, (2) to choose individual innovati<strong>on</strong> metrics, and (3) to build a portfolio of metrics.<br />

Table 4: Questi<strong>on</strong>s to guide the choice of innovati<strong>on</strong> metrics<br />

Questi<strong>on</strong> Framework Element Comment<br />

Why<br />

What<br />

How<br />

Where/<br />

When/<br />

Who<br />

References<br />

The Goals of<br />

innovating<br />

The Resources<br />

required and the<br />

anticipated Results<br />

The Process of<br />

innovating<br />

Are we doing the right things? What are the organisati<strong>on</strong>’s goals for<br />

innovati<strong>on</strong>? Ultimately the metrics are required to ensure that goals<br />

are being met. This is a questi<strong>on</strong> of aligning metrics with the goals.<br />

Are we using the correct resources in order to produce the right<br />

results? What are the resources required in order to achieve each<br />

goal and what are the individual results necessary to achieve each<br />

goal? This is a questi<strong>on</strong> of aligning metrics with the inputs and<br />

outputs.<br />

Are we doing things right? How does the organisati<strong>on</strong> intend to<br />

achieve each goal? This is a questi<strong>on</strong> of aligning metrics with the<br />

process.<br />

And in the right place? Within which c<strong>on</strong>text does the organisati<strong>on</strong><br />

innovate? This is a questi<strong>on</strong> of aligning the metrics with the<br />

structure and c<strong>on</strong>text.<br />

Adams, R., J. Bessant and R. Phelps. 2006. "Innovati<strong>on</strong> management measurement: A review." Internati<strong>on</strong>al<br />

Journal of <strong>Management</strong> Reviews 8(1):21-47.<br />

249


Paidi O’Raghallaigh, David Samm<strong>on</strong> and Ciaran Murphy<br />

Alves, J., M. J. Marques, I. Saur and P. Marques. 2007. "Creativity and innovati<strong>on</strong> through multidisciplinary and<br />

multisectoral cooperati<strong>on</strong>." Creativity and Innovati<strong>on</strong> <strong>Management</strong> 16(1):27-34.<br />

Amabile, T. M. 1999. "How to Kill Creativity." Harvard Business Review 76(5).<br />

Archibugi, D. and M. Planta. 1996. "Measuring technological change through patents and innovati<strong>on</strong> surveys."<br />

Technovati<strong>on</strong> 16(9):451-468.<br />

Arundel, A., K. Smith, P. Patel and G. Sirilli. 1998. "The future of innovati<strong>on</strong> measurement in Europe. C<strong>on</strong>cepts,<br />

problems and practical directi<strong>on</strong>s." Step Group.<br />

Becheikh, N., R. Landry and N. Amara. 2006. "Less<strong>on</strong>s from innovati<strong>on</strong> empirical studies in the manufacturing<br />

sector: A systematic review of the literature from 1993-2003." Technovati<strong>on</strong> 26(5-6):644-664.<br />

Böhme, R. and F. Freiling. 2008. "On metrics and measurements." Dependability metrics:7-13.<br />

Cagan, J. and C. M. Vogel. 2002. Creating breakthrough products: innovati<strong>on</strong> from product planning to program<br />

approval: FT Press.<br />

Carroll, S. J. and D. J. Gillen. 1987. "Are the classical management functi<strong>on</strong>s useful in describing managerial<br />

work?" Academy of <strong>Management</strong> Review 12(1):38-51.<br />

Cassiman, B. and R. Veugelers. 2002. "Spillovers and R&D cooperati<strong>on</strong>: some empirical evidence." American<br />

Ec<strong>on</strong>omic Review 92(4):1169–1184<br />

Christensen, C. M. 2006. Using Disruptive Innovati<strong>on</strong> to Create New Growth: J. Willard Marriott Library,<br />

University of Utah.<br />

Collins, J. and D. Smith. 1999. "Innovati<strong>on</strong> metrics: A framework to accelerate growth." PRISM-CAMBRIDGE<br />

MASSACHUSETTS-:33-48.<br />

Daft, R. 1998. "Organisati<strong>on</strong> Theory and Design Internati<strong>on</strong>al." <str<strong>on</strong>g>6th</str<strong>on</strong>g> Editi<strong>on</strong> Editi<strong>on</strong>. Cincinnati, Ohio, US:<br />

Thomps<strong>on</strong> Publishing.<br />

Davila, A., G. Foster and M. Li. 2009. "Reas<strong>on</strong>s for management c<strong>on</strong>trol systems adopti<strong>on</strong>: Insights from product<br />

development systems choice by early-stage entrepreneurial companies." Accounting, Organizati<strong>on</strong>s and<br />

Society 34(3-4):322-347.<br />

Ditillo, A. 2004. "Dealing with uncertainty in knowledge-intensive firms: the role of management c<strong>on</strong>trol systems<br />

as knowledge integrati<strong>on</strong> mechanisms* 1." Accounting, Organizati<strong>on</strong>s and Society 29(3-4):401-421.<br />

Hagedoorn, J. and M. Cloodt. 2003. "Measuring innovative performance: is there an advantage in using multiple<br />

indicators?" Research Policy 32(8):1365–1379<br />

Hansen, M. T. and J. Birkinshaw. 2007. "The innovati<strong>on</strong> value chain." Harvard Business Review 85(6):121.<br />

Hoch, D. J., C. Roeding, S. K. Lindner and G. Purkert. 2000. Secrets of software success: Harvard Business<br />

School Press Bost<strong>on</strong>.<br />

Jaworski, B. J. 1988. "Toward a theory of marketing c<strong>on</strong>trol: envir<strong>on</strong>mental c<strong>on</strong>text, c<strong>on</strong>trol types, and<br />

c<strong>on</strong>sequences." The Journal of Marketing:23-39.<br />

Kandybin, A. and M. Kihn. 2004. "Raising your return <strong>on</strong> innovati<strong>on</strong> investment." Strategy And<br />

Business(Spring):38-49.<br />

Kerssens van Dr<strong>on</strong>gelen, I. and J. Bilderbeek. 1999. "R&D performance measurement: more than choosing a set<br />

of metrics." R&D <strong>Management</strong> 29(1):35-46.<br />

Koen, P. A. 2004. "The Fuzzy Fr<strong>on</strong>t End for Incremental, Platform, and Breakthrough Products." The PDMA<br />

Handbook of New Product Development, John Wiley & S<strong>on</strong>s, New York:81–91.<br />

Koen, P., G. Ajamian, R. Burkart, A. Clamen, J. Davids<strong>on</strong>, R. D'Amore, C. Elkins, K. Herald, M. Incorvia and A.<br />

Johns<strong>on</strong>. 2001. "Providing clarity and a comm<strong>on</strong> language to the" fuzzy fr<strong>on</strong>t end"." Research-Technology<br />

<strong>Management</strong> 44(2):46-55.<br />

Kogut, B. and U. Zander. 1996. "What firms do? Coordinati<strong>on</strong>, identity, and learning." Organizati<strong>on</strong> Science<br />

7(5):502-518.<br />

Linder, J. C. 2006. "Does innovati<strong>on</strong> drive profitable growth? New metrics for a complete picture." Journal of<br />

Business Strategy 27(5):38-44.<br />

Mathiassen, L. and A. M. Vainio. 2007. "Dynamic capabilities in small software firms: A sense-and-resp<strong>on</strong>d<br />

approach." Engineering <strong>Management</strong>, IEEE Transacti<strong>on</strong>s <strong>on</strong> 54(3):522-538.<br />

Mockler, R. J. 1970. Readings in management c<strong>on</strong>trol: Applet<strong>on</strong>-Century-Crofts and Fleschner Publishing<br />

Company.<br />

Muller, A., L. Välikangas and P. Merlyn. 2005. "Metrics for innovati<strong>on</strong>: guidelines for developing a customized<br />

suite of innovati<strong>on</strong> metrics." Strategy & Leadership 33(1):37-45.<br />

Nambisan, S. 2002. "Software firm evoluti<strong>on</strong> and innovati<strong>on</strong>-orientati<strong>on</strong>." Journal of Engineering and Technology<br />

<strong>Management</strong> 19(2):141-165.<br />

O'Raghallaigh, P., D. Samm<strong>on</strong> and C. Murphy. 2010. "Theory-building using Typologies - A Worked Example of<br />

Building a Typology of Knowledge Activities for Innovati<strong>on</strong>." In 15th IFIP WG8.3 <str<strong>on</strong>g>C<strong>on</strong>ference</str<strong>on</strong>g> <strong>on</strong> DSS, ed.<br />

Various. University of Lisb<strong>on</strong>, Portugal: IOS Press.<br />

Osterwalder, A., Y. Pigneur and C. L. Tucci. 2005. "Clarifying business models: Origins, present, and future of<br />

the c<strong>on</strong>cept." Communicati<strong>on</strong>s of the associati<strong>on</strong> for Informati<strong>on</strong> Systems 16(1):1-25.<br />

Sawhney, M., R. C. Wolcott and I. Arr<strong>on</strong>iz. 2006. "The 12 different ways for companies to innovate." MIT Sloan<br />

<strong>Management</strong> Review 47(3):75.<br />

Shapiro, A. R. 2006. "Measuring innovati<strong>on</strong>: bey<strong>on</strong>d revenue from new products." Research-Technology<br />

<strong>Management</strong> 49(6):42-51.<br />

Smith, K. 1998. Science, technology and innovati<strong>on</strong> indicators: a guide for policymakers: STEP Group.<br />

250


Using Focus Groups to Evaluate Artefacts in Design<br />

Research<br />

Paidi O’Raghallaigh, David Samm<strong>on</strong> and Ciaran Murphy<br />

Business Informati<strong>on</strong> Systems, University College Cork, Cork, Ireland<br />

paidioreilly@gmail.com<br />

dsamm<strong>on</strong>@afis.ucc.ie<br />

cmurphy@afis.ucc.ie<br />

Abstract: Evaluati<strong>on</strong> in design research c<strong>on</strong>tinues to be ad hoc and poorly performed. It is <strong>on</strong>e of the single<br />

biggest weaknesses in existing design research. Part of the problem is undoubtedly disagreement around the<br />

nature of design research and the highly complex process around evaluating its scientific claims. These issues<br />

demand our collective attenti<strong>on</strong>. This paper proposes that evaluati<strong>on</strong> in design research must answer two key<br />

questi<strong>on</strong>s regarding the artefact under c<strong>on</strong>siderati<strong>on</strong> - ‘does the artefact work’ and ‘why does it work’. This paper<br />

moves bey<strong>on</strong>d the traditi<strong>on</strong>al approach to evaluati<strong>on</strong> of artefacts and instead describes an interacti<strong>on</strong>ist<br />

approach. Focus groups are proposed as an appropriate method for answering the above questi<strong>on</strong>s. Guidelines<br />

for the use of focus groups as an interacti<strong>on</strong>ist approach to evaluati<strong>on</strong> are provided. Up to now the use of focus<br />

group methods to evaluate and refine design artefacts has remained relatively new to the IS field.<br />

Keywords: design science, design research, design, evaluati<strong>on</strong>, focus groups<br />

1. Introducti<strong>on</strong><br />

March and Smith (1995) identify the purposeful building of artefacts and the subsequent evaluating of<br />

those artefacts as two main tasks in design research. The artefacts have to be evaluated in order to<br />

c<strong>on</strong>clude if any progress has been made. While evaluati<strong>on</strong> remains a headline issue in design<br />

research and there c<strong>on</strong>tinues to be a need for rigorous evaluati<strong>on</strong> methods (Hevner et al. 2004), there<br />

remains little guidance in the literature c<strong>on</strong>cerning what is rigorous evaluati<strong>on</strong> and how to choose and<br />

design a rigorous evaluati<strong>on</strong> strategy (Pries-Heje et al. 2008). There are few widely-shared<br />

frameworks to guide how to make reas<strong>on</strong>ed choices when planning an evaluati<strong>on</strong> design (Mark<br />

1999). Instead evaluators from different camps bring vastly different views of what type of evaluati<strong>on</strong><br />

should be d<strong>on</strong>e in a given situati<strong>on</strong> (ibid). This results in a seemingly unlimited number of approaches<br />

and evaluators are having trouble knowing what is what (Mark 1999).<br />

Despite calls for more rigorous design research, evaluati<strong>on</strong> c<strong>on</strong>tinues to be ad hoc rather than a<br />

systematic practice (Avgerou 1995). The result is that scholars struggle with evaluati<strong>on</strong>, which is<br />

poorly performed (Pries-Heje et al. 2008). Venable (2010) observes how evaluati<strong>on</strong> is <strong>on</strong>e of the<br />

single biggest weaknesses in existing design research. Part of the problem is undoubtedly the highly<br />

complex process that results from “… the various interrelated factors that have to be c<strong>on</strong>sidered when<br />

carrying out an evaluati<strong>on</strong>” (Cleven et al. 2009, np). This complexity demands a ‘structured<br />

proceeding’ towards designing an evaluati<strong>on</strong> strategy (Cleven et al. 2009; Avgerou 1995). The<br />

objective of this paper is to investigate some of the issues facing the design research scholar and to<br />

propose an interacti<strong>on</strong>ist approach to evaluati<strong>on</strong> with design theory at its core. This is followed with a<br />

detailed look at focus groups as an appropriate interacti<strong>on</strong>ist method.<br />

While focus groups are now <strong>on</strong>e of the most widely used research tools in the social sciences<br />

(Tremblay et al. 2010), relatively little was published about them until more recent years (Rezabek<br />

2000). Indeed the use of focus group methods to evaluate and refine artefacts remains relatively new<br />

to the IS field (Tremblay et al. 2010). In additi<strong>on</strong>, the majority of the literature that is available tends to<br />

be written by n<strong>on</strong>-designers (McD<strong>on</strong>agh-Philp et al. 2000). The objective of this secti<strong>on</strong> is, therefore,<br />

to investigate the use of face-to-face and computer-mediated focus groups as approaches to<br />

interacti<strong>on</strong>ist evaluati<strong>on</strong> for design research.<br />

2. Background to design research<br />

It is necessary to distinguish between the practice of design and the science of design. Both are<br />

problem-solving activities but the difference lies in their c<strong>on</strong>tributi<strong>on</strong>s to the body of design knowledge.<br />

Artefact c<strong>on</strong>structi<strong>on</strong> through applying existing knowledge is the prime focus of design practice while<br />

knowledge generati<strong>on</strong> through artefact c<strong>on</strong>structi<strong>on</strong> is paramount for design research (Niehaves<br />

2007). However, there are serious misgivings am<strong>on</strong>g scholars as to the required nature of this<br />

knowledge. This paper argues that design science must generate knowledge of a theoretical nature<br />

251


Paidi O’Raghallaigh, David Samm<strong>on</strong> and Ciaran Murphy<br />

that explains both the how and why of artefact c<strong>on</strong>structi<strong>on</strong>. The design knowledge makes<br />

c<strong>on</strong>tributi<strong>on</strong>s to the academic knowledge base in the form of design theories (Walls et al. 1992).<br />

Design theories give explicit prescripti<strong>on</strong>s <strong>on</strong> how to design an artefact and in additi<strong>on</strong>, they draw <strong>on</strong><br />

kernel theories in order to explain why a design should work (ibid). Iivari (2007, p. 49) c<strong>on</strong>siders “…<br />

the existence of a kernel theory to be a defining characteristic of a ‘design theory’ … [and] without a<br />

sound kernel theory it is not justified to speak about ‘design theory’”. In summary the primary role of<br />

design research is, therefore, the generati<strong>on</strong> of design theories and the corresp<strong>on</strong>ding kernel<br />

theories, while the emerging artefact is no more than a sec<strong>on</strong>dary (albeit a necessary) output of the<br />

research. However, we now look at the implicati<strong>on</strong>s this has <strong>on</strong> how design research is evaluated.<br />

2.1 Evaluati<strong>on</strong> in the design research literature<br />

Design research c<strong>on</strong>sists of activities to design an innovative artefact for a specific purpose and to<br />

subsequently evaluate how well it performs in relati<strong>on</strong> to this purpose (March and Smith 1995). These<br />

activities are typically iterated a number of times before the final artefact is arrived at (Markus et al.<br />

2002). Here evaluati<strong>on</strong> provides evidence of “… how well the artifact supports a soluti<strong>on</strong> to the<br />

problem” (Peffers et al. 2007, p. 56). March and Smith, (1995, p. 258) express this state of affairs as<br />

follows: “We build an artifact to perform a specific task. The basic questi<strong>on</strong> is, does it work? .... We<br />

evaluate artifacts to determine if we have made any progress”.<br />

But this paper claims that <strong>on</strong> its own this is insufficient for a scientific c<strong>on</strong>tributi<strong>on</strong>. A clear distincti<strong>on</strong><br />

must be made between, <strong>on</strong> the <strong>on</strong>e hand, evaluati<strong>on</strong> focused <strong>on</strong> the artefact and its utility, and, <strong>on</strong> the<br />

other hand, evaluati<strong>on</strong> focused <strong>on</strong> the resulting theory and the explanati<strong>on</strong>s it provides. The former<br />

addresses the questi<strong>on</strong>s of does the artefact work and how well does it work. The latter addresses the<br />

questi<strong>on</strong>s of why does the artefact work and why does it work so well. In other words evaluati<strong>on</strong> of an<br />

artefact should c<strong>on</strong>tribute observati<strong>on</strong>-based insights that improve the explanatory functi<strong>on</strong> of the<br />

design theory but it should also c<strong>on</strong>tribute observati<strong>on</strong>-based justificati<strong>on</strong>s for the explanati<strong>on</strong>s offered<br />

by the design theory. This dovetails nicely with the work of Goldkuhl and Lind (2010), who advocate<br />

that design theories should be justified through both empirical and theoretical groundings. Now we<br />

turn our attenti<strong>on</strong> to addressing an issue related to how evaluati<strong>on</strong> is performed.<br />

2.2 An interacti<strong>on</strong>ist approach to evaluati<strong>on</strong><br />

Evaluati<strong>on</strong>, including that in the field of IS, has been dominated by overly positivistic and scientific<br />

paradigms (Serafeimidis and Smiths<strong>on</strong> 2003). For instance, March and Smith (1995, p. 258) state<br />

that: “Evaluati<strong>on</strong> requires the development of metrics and the measurement of artifacts according to<br />

those metrics. Metrics define what we are trying to accomplish. They are used to assess the<br />

performance of an artifact. Lack of metrics and failure to measure artifact performance according to<br />

established criteria result in an inability to effectively judge research efforts”. The result is a highly<br />

formal and rati<strong>on</strong>al approach that views evaluati<strong>on</strong> (for the most part) as reaching an objective<br />

judgement <strong>on</strong> the techno-elements of a discrete system while ignoring the c<strong>on</strong>siderable socioelements<br />

that are part of the wider c<strong>on</strong>cepti<strong>on</strong> of many systems (Sym<strong>on</strong>s, 1991). As a result, many<br />

scholars (Barrow and Mayhew 2000; Guba and Lincoln 1989; Iivari 1988; Smiths<strong>on</strong> and Hirschheim<br />

1998; Sym<strong>on</strong>s 1991) are moved to argue that evaluati<strong>on</strong> should instead adopt a more interacti<strong>on</strong>ist<br />

approach that strives to incorporate various stakeholder interests and perspectives when determining<br />

the value of socially embedded systems. Avgerou (1995) advocates that evaluati<strong>on</strong> should involve<br />

extensive stakeholder participati<strong>on</strong> in a dialectic process, where the aspects of the artefact to be<br />

evaluated, as well as the criteria to be applied in the evaluati<strong>on</strong>, are those deemed important and<br />

emerge from the c<strong>on</strong>cerns, claims and views expressed by the stakeholder groups. Such qualitative<br />

evaluati<strong>on</strong> has a better possibility to describe why goals and criteria are fulfilled . An interacti<strong>on</strong>ist<br />

approach, therefore, seems sensible when evaluating the c<strong>on</strong>tributi<strong>on</strong> of design research to<br />

answering questi<strong>on</strong>s of why. Next we look at how an interacti<strong>on</strong>ist approach might be implemented.<br />

3. An evaluati<strong>on</strong> approach using focus groups<br />

Much of the emphasis in qualitative research during the 20th century was <strong>on</strong> participant observati<strong>on</strong><br />

and individual interviews (Rezabek 2000). But falling in the c<strong>on</strong>tinuum of qualitative research between<br />

these methods, focus groups c<strong>on</strong>sist of semi-structured questi<strong>on</strong> sessi<strong>on</strong>s in which a moderator<br />

promotes interacti<strong>on</strong> am<strong>on</strong>g a collecti<strong>on</strong> of participants that have been brought together to discuss<br />

and shed light <strong>on</strong> a particular topic, issue or c<strong>on</strong>cern (Hansen and Hansen 2006; McD<strong>on</strong>agh-Philp et<br />

al. 2000; Powell and Single 1996; Rezabek 2000; Tremblay et al. 2010). For example, Tremblay et al.<br />

(Tremblay et al. 2010, p. 600) state that focus groups are “… valuable to gain shared understandings<br />

252


Paidi O’Raghallaigh, David Samm<strong>on</strong> and Ciaran Murphy<br />

but yet allows for individual differences of opini<strong>on</strong> to be voiced”. The key characteristic which<br />

distinguishes focus groups from other approaches is the level of interacti<strong>on</strong> between participants and<br />

the synergy within the group (Gibbs 1997; Kitzinger 1995). The unit of analysis is, therefore, mainly<br />

(but not exclusively) at the group level rather than the individual participant level (Krueger and Casey<br />

2000).<br />

3.1 Strengths and weaknesses of focus groups<br />

Drawing <strong>on</strong> this discussi<strong>on</strong>, as well as the work of Gibbs (1997), McD<strong>on</strong>agh-Philp and Bruseberg<br />

(2000), Bruseberga and McD<strong>on</strong>agh-Philp (2002), K<strong>on</strong>tio et al. (2004), Mazza, and Berrè (2007), and<br />

Tremblay et al. (2010), the strengths of focus groups as a research method are outlined in Table 1.<br />

Table 1: Main strengths of focus groups<br />

Strength Descripti<strong>on</strong><br />

Fast and cost Because several subjects can be ‘interviewed’ at the same time, focus groups are a<br />

effective method fast and cost effective means of obtaining attitudes, feelings, and beliefs.<br />

Provides broad Focus groups allow for an open format and they are flexible enough to handle a<br />

c<strong>on</strong>tent<br />

wide range of topics.<br />

Provides in-depth Focus groups allow in-depth explorati<strong>on</strong> of the reas<strong>on</strong>s why the participants think<br />

c<strong>on</strong>tent the way they do and often provide insights that can be difficult, time c<strong>on</strong>suming, or<br />

expensive to capture using other methods.<br />

Builds new c<strong>on</strong>tent Focus groups provide participants with the opportunity to react to, reflect <strong>on</strong>, and<br />

through interacti<strong>on</strong> build <strong>on</strong> the experiences of others. This can generate new ideas that might have<br />

not been uncovered in individual interviews. It also provides the scholar with the<br />

opportunity to clarify resp<strong>on</strong>ses, ask follow-up questi<strong>on</strong>s, and receive c<strong>on</strong>tingent<br />

answers to questi<strong>on</strong>s.<br />

Empowers The opportunity to work collaboratively with scholars can be empowering for many<br />

participants participants. In additi<strong>on</strong>, participants build <strong>on</strong> their own knowledge through gaining<br />

from other participants, benchmarking experiences and practices between<br />

companies, and increasing networking c<strong>on</strong>tacts.<br />

Drawing from the same sources the weaknesses of focus groups as a research method are outlined<br />

in Table 2.<br />

Table 2: Main weaknesses of focus groups<br />

Weakness Descripti<strong>on</strong><br />

Difficult to assemble Focus groups can be difficult to assemble and to get representative samples to<br />

work with. In additi<strong>on</strong>, certain people, such as those who are not very articulate or<br />

c<strong>on</strong>fident, and those who have communicati<strong>on</strong> problems or special needs, may be<br />

Loss of c<strong>on</strong>trol over<br />

the data produced<br />

Resp<strong>on</strong>ses from group<br />

members are not<br />

independent of <strong>on</strong>e<br />

other<br />

Results are not fully<br />

c<strong>on</strong>fidential or<br />

an<strong>on</strong>ymous<br />

discouraged from participating.<br />

Other than keeping participants focused <strong>on</strong> the topic, the scholar has less c<strong>on</strong>trol<br />

over the data produced as participants must be allowed to talk to each other, ask<br />

questi<strong>on</strong>s, and express doubts and opini<strong>on</strong>s.<br />

It cannot be assumed that participants are expressing their pers<strong>on</strong>al views but<br />

instead they may be expressing those that they perceive to be most acceptable to<br />

the group. In additi<strong>on</strong>, a limited number of participants may dominate proceeding<br />

and bias the result, while more reserved participants are hesitant to express their<br />

views.<br />

The discussi<strong>on</strong>s taking place in focus groups are never fully c<strong>on</strong>fidential or<br />

an<strong>on</strong>ymous because the c<strong>on</strong>tent is being shared with others in the group. Some<br />

participants may, therefore, decide to withhold some relevant informati<strong>on</strong> because<br />

of c<strong>on</strong>fidentiality c<strong>on</strong>cerns.<br />

Limited generalisability Because of difficulties in putting together a representative sample, getting all<br />

participants heard, and also due to some participants possibly having hidden<br />

Analysis of the data<br />

can be difficult<br />

Time c<strong>on</strong>straints and<br />

complexity of<br />

discussi<strong>on</strong>s<br />

Requirement for a<br />

skilled and<br />

experienced<br />

moderator<br />

agendas the results may be biased and it may be difficult to generalise from them.<br />

The open-ended nature of the questi<strong>on</strong>s and the wide ranging resp<strong>on</strong>ses can<br />

make the analysis of the data difficult.<br />

The time available for discussi<strong>on</strong>s in a focus group is limited and this means that<br />

some complex issues may not receive sufficient time and may not be covered<br />

appropriately.<br />

A skilled and experienced moderator is needed for an effective research study<br />

253


Paidi O’Raghallaigh, David Samm<strong>on</strong> and Ciaran Murphy<br />

A review of the role of focus groups in design research is now provided.<br />

4. Focus groups within design research<br />

Focus groups are suited to design research in that they offer designers a flexible range of techniques<br />

that can be applied at the various stages of an iterative design process, thereby providing a c<strong>on</strong>sistent<br />

and encompassing research method supporting the full design process (Nielsen 1997). Two distinct<br />

forms of artefact evaluati<strong>on</strong> are generally performed within a design initiative – exploratory evaluati<strong>on</strong><br />

that takes place during the build/evaluate design cycle to clarify the stakeholder needs and to refine<br />

the design of the artefact, and c<strong>on</strong>firmatory evaluati<strong>on</strong> that takes place after the design cycle for the<br />

field testing of an instantiati<strong>on</strong> of the artefact in a particular envir<strong>on</strong>ment (Hevner 2007). Tremblay et<br />

al. (2010), therefore, propose two distinct types of focus groups for design research: exploratory focus<br />

groups (EFG), which are used for the rapid incremental design and refinement of an artefact and<br />

c<strong>on</strong>firmatory focus groups (CFG), which are used to provide c<strong>on</strong>firmatory evidence of an artefact’s<br />

utility in the field.<br />

But there is little clarity in the literature regarding an appropriate number of focus group sessi<strong>on</strong>s. At<br />

the minimum, Tremblay et al. (2010) suggest that there should be <strong>on</strong>e pilot focus group, two EFGs,<br />

and at least two CFGs. The pilot sessi<strong>on</strong> is informal and used “… to understand timing issues and any<br />

kinks in the questi<strong>on</strong>ing route” . There should be at least two design cycles whereby each is driven by<br />

the findings from an EFG. Finally, as “… the unit of analysis is the focus group, it would be difficult to<br />

make a compelling argument for the utility of the designed artifact with just <strong>on</strong>e CFG” and they,<br />

therefore, recommend at least two . There is also a lack of transparency in the literature regarding the<br />

appropriate number of focus group participants. The lower boundary for the number of participants is<br />

about four and the upper boundary is twelve (Tremblay et al. 2010). While it can seem easier (and<br />

less expensive) to divide participants into fewer but larger focus groups, this lowers the sample size<br />

as there are then fewer groups across which to compare results . In additi<strong>on</strong>, the dynamics of larger<br />

groups tend to be very different to that of smaller groups in that less interacti<strong>on</strong> is required from each<br />

participant with the result that larger groups can lead to social loafing . On the other hand, a diversity<br />

of participants usually triggers more creative and broad discussi<strong>on</strong>s (and perhaps more c<strong>on</strong>flict), but<br />

segregati<strong>on</strong> of participants based <strong>on</strong> skills and knowledge may provide more in-depth discussi<strong>on</strong>s .<br />

Unfortunately, the procedural literature about focus group is very light in detail regarding the c<strong>on</strong>duct<br />

of focus groups and it is not tailored to the specific needs of designers or design research (Bruseberg<br />

and McD<strong>on</strong>agh-Philp 2002). We now examine how focus groups may be planned and run.<br />

4.1 Planning and organising focus groups<br />

The workload and resp<strong>on</strong>sibility <strong>on</strong> the moderator and facilitator can be <strong>on</strong>erous and the pre-planning<br />

of focus groups is, therefore, critical to their success. Such planning can take weeks and even m<strong>on</strong>ths<br />

of preparati<strong>on</strong> effort. Due to the open-ended nature of focus groups, moderati<strong>on</strong> can be complex and<br />

Krueger et al. (2000) identify the following attributes that they deem important when moderating a<br />

focus group: (1) presenting a friendly manner and a sense of humour, (2) involving and allowing all<br />

participants the opportunity to express their views, (3) challenging participants to draw out differences<br />

in opini<strong>on</strong>s and to tease out a diverse range of meanings, (4) communicating clearly, both orally and<br />

in writing, and (5) listening to the views of others, while c<strong>on</strong>trolling pers<strong>on</strong>al views. In additi<strong>on</strong>, the<br />

moderator should have a clear understanding of various aspects of the artefact being evaluated and<br />

be comfortable presenting it to focus group participants (Tremblay et al. 2010).<br />

The questi<strong>on</strong>ing route is also central to the success of the focus groups and ought to be closely<br />

aligned with the research problem being addressed. Tremblay et al. (2010) recommend that there be<br />

no more than twelve questi<strong>on</strong>s for a two-hour sessi<strong>on</strong> and that the questi<strong>on</strong>s be ordered, firstly, from<br />

the most general to the more specific and, sec<strong>on</strong>dly, by the relative importance of the questi<strong>on</strong> to the<br />

research agenda. So for a given artefact, they suggest beginning with an explanati<strong>on</strong> of its purpose,<br />

followed by an outline of different scenarios in which it could be utilised, a descripti<strong>on</strong> of its design,<br />

training <strong>on</strong> its use, and finishing with a task where focus group participants are asked to utilise and<br />

evaluate the artefact. A promising evaluati<strong>on</strong> approach can be to use an ‘exercise’ within the focus<br />

group, whereby participants are asked to collectively complete a task without the artefact and again<br />

with the artefact. The ensuing discussi<strong>on</strong> should revolve around how the artefact was used and how<br />

the completi<strong>on</strong> of the task was altered by its use.<br />

254


Paidi O’Raghallaigh, David Samm<strong>on</strong> and Ciaran Murphy<br />

Drawing from this discussi<strong>on</strong>, Table 3 summarises the steps that should be c<strong>on</strong>sidered when<br />

organising focus groups for design research purposes.<br />

Table 3: Steps for organising focus groups in design research (after: Gibbs 1997; Gibs<strong>on</strong> and Arnott<br />

2007; Tremblay et al. 2010)<br />

Step Descripti<strong>on</strong><br />

Defining the<br />

research<br />

problem and<br />

the type of<br />

focus group.<br />

Determining<br />

the number<br />

and the<br />

durati<strong>on</strong> of<br />

focus groups<br />

Determining<br />

the number<br />

and type of<br />

participants in<br />

each focus<br />

group sessi<strong>on</strong><br />

Identifying the<br />

moderator<br />

Identifying a<br />

questi<strong>on</strong>ing<br />

route<br />

The appropriateness of the two types of focus group - exploratory focus groups (EFG) and<br />

c<strong>on</strong>firmatory focus groups (CFG) - depends <strong>on</strong> the research goals of the design initiative.<br />

The series of focus groups should c<strong>on</strong>tinue until nothing new is being learned from further<br />

sessi<strong>on</strong>s. But this is an extremely difficult and arbitrary decisi<strong>on</strong> and instead the scholar<br />

may need to accept that the time has come to move forward even though something new<br />

can always be learned from further sessi<strong>on</strong>s.<br />

The identificati<strong>on</strong> of representative, insightful and motivated participants is critical to the<br />

success of focus groups. Putting a group together should not involve a random selecti<strong>on</strong>,<br />

but should instead be based <strong>on</strong> the characteristics of the participants in relati<strong>on</strong> to the<br />

research problem and the soluti<strong>on</strong> being evaluated. The lower boundary for the number of<br />

participants is about four and the upper boundary twelve.<br />

Due to the open-ended nature of focus groups, moderati<strong>on</strong> is complex and places<br />

demands <strong>on</strong> the required pers<strong>on</strong>ality, nature, and abilities required of a moderator. In<br />

additi<strong>on</strong>, the moderator should have a clear understanding of various aspects of the<br />

artefact being evaluated and be comfortable presenting it to focus group participants.<br />

The questi<strong>on</strong>ing route is the agenda for the focus group and sets the broad directi<strong>on</strong> for<br />

the group discussi<strong>on</strong> and it, therefore, should be closely aligned with the research<br />

problem.<br />

Once all the planning and organising of the focus group has taken place, the day arrives when a focus<br />

group sessi<strong>on</strong> needs to take place.<br />

4.2 Running the focus group sessi<strong>on</strong>s<br />

A typical focus group sessi<strong>on</strong> can last about two hours and generally c<strong>on</strong>sists of the moderator<br />

guiding anywhere between four to twelve people through a focused discussi<strong>on</strong> of a specific topic.<br />

During this time the moderator must expertly ensure that “… the focus-group sessi<strong>on</strong> should feel freeflowing<br />

and relatively unstructured, but in reality, the moderator must follow a preplanned script of<br />

specific issues and set goals for the type of informati<strong>on</strong> to be gathered. During the group sessi<strong>on</strong>, the<br />

moderator has the difficult job of keeping the discussi<strong>on</strong> <strong>on</strong> track without inhibiting the flow of ideas<br />

and comments. The moderator also must ensure that all group members c<strong>on</strong>tribute to the discussi<strong>on</strong><br />

and must avoid letting <strong>on</strong>e participant’s opini<strong>on</strong>s dominate” (Nielsen 1997, p. 95). The seating<br />

arrangements for (face-to-face) focus group sessi<strong>on</strong>s can be important. Tremblay et al. (2010)<br />

recommend that a good approach may be to get to know the participants before the questi<strong>on</strong>ing route<br />

begins and to place the participants in an U-shape arrangement with the most assertive and expert<br />

participants next to the moderator, while the least talkative are seated directly across from the<br />

moderator. During the sessi<strong>on</strong> new topics may emerge requiring the moderator to think <strong>on</strong> his or her<br />

feet and ask further probing follow-up questi<strong>on</strong>s of the participants, while at the same time not losing<br />

sight of the focus of the sessi<strong>on</strong> (Rezabek 2000). In additi<strong>on</strong>, the moderator may need to encourage<br />

some participants to c<strong>on</strong>tribute to the sessi<strong>on</strong>, while at the same time safeguarding against assertive<br />

participants dominating the c<strong>on</strong>versati<strong>on</strong>.<br />

Focus groups sessi<strong>on</strong>s can be recorded using video and/or audio tapes. Other times an observer<br />

might be used to take notes of exchanges and also to record any str<strong>on</strong>g reacti<strong>on</strong>s, facial expressi<strong>on</strong>s,<br />

and/or general t<strong>on</strong>e in the interacti<strong>on</strong>s (Tremblay et al. 2010). In some cases “… data analysis can be<br />

as simple as having the moderator write a short report summing up the prevailing mood in the group,<br />

illustrated with a few colorful quotes. You can also do more detailed analyses, but the unstructured<br />

nature of the groups make this difficult and time c<strong>on</strong>suming” (Nielsen 1997, p. 95). When analysing<br />

and reporting the results, the c<strong>on</strong>tents of the discussi<strong>on</strong>s should be examined for their meanings and<br />

their implicati<strong>on</strong>s for the research questi<strong>on</strong>s. Scholars should look for comm<strong>on</strong> themes and variati<strong>on</strong>s<br />

within the transcripts that provide rich descripti<strong>on</strong>s of the participants’ reacti<strong>on</strong>s to design features.<br />

255


Paidi O’Raghallaigh, David Samm<strong>on</strong> and Ciaran Murphy<br />

Short quotes may be used to aid in the specific points of interpretati<strong>on</strong> and l<strong>on</strong>ger passages of<br />

quotati<strong>on</strong> can be used to give a flavour of the original discussi<strong>on</strong>s. Summary tables can be very<br />

helpful, displaying both evidence and counter-evidence of the utility of the soluti<strong>on</strong> by focus group.<br />

Drawing from this discussi<strong>on</strong>, Table 4 outlines the steps that should be c<strong>on</strong>sidered when moderating,<br />

recording and reporting <strong>on</strong> focus groups.<br />

Table 4: Steps for moderating, recording, and reporting focus groups in design research (after: Gibbs<br />

1997; Gibs<strong>on</strong> and Arnott 2007; Tremblay et al. 2010)<br />

Step Descripti<strong>on</strong><br />

C<strong>on</strong>ducting<br />

the focus<br />

group sessi<strong>on</strong><br />

Capturing and<br />

recording the<br />

focus group<br />

sessi<strong>on</strong><br />

Analysing and<br />

reporting the<br />

focus group<br />

sessi<strong>on</strong><br />

The focus group sessi<strong>on</strong> must be carefully managed for time while still ensuring that all<br />

main c<strong>on</strong>tributi<strong>on</strong>s can be made during the allocated time.<br />

Relying <strong>on</strong> moderator notes may not be sufficient as being a moderator is a full-time job<br />

in a focus group sessi<strong>on</strong>. Focus groups may be video and/or audio taped.<br />

The methods for analysing the focus group data have many of the same challenges in<br />

dem<strong>on</strong>strating rigor that all qualitative research encounters share. Techniques that are<br />

used for qualitative data analysis and that emphasise the reliability and replicability of<br />

the observati<strong>on</strong>s and results should be c<strong>on</strong>sidered and used where appropriate.<br />

While much of this discussi<strong>on</strong> has assumed that focus groups involve all participants being located in<br />

the same physical space and c<strong>on</strong>tributing at the same time, this is not always the case. An example is<br />

computer-mediated focus groups, which can demand some necessary changes in how the focus<br />

groups are moderated and facilitated.<br />

5. C<strong>on</strong>clusi<strong>on</strong>s<br />

Despite the recent emphasis <strong>on</strong> the importance of evaluati<strong>on</strong> in design research, it c<strong>on</strong>tinues to be an<br />

ad hoc rather than a systematic practice. Evaluati<strong>on</strong> in design research becomes even more complex<br />

when the positi<strong>on</strong> of theory is taken into c<strong>on</strong>siderati<strong>on</strong>. This study adopts the positi<strong>on</strong> that creating<br />

novel or effective artefacts without complementary theory-building is neither rigorous nor scientific.<br />

According to this view, design research must generate abstract design knowledge about artefacts in<br />

the form of design theories and that artefacts are no more than ‘tests’ of design theories. Evaluati<strong>on</strong><br />

must fill two roles in that it must c<strong>on</strong>currently address the questi<strong>on</strong>s of ‘does the artefact work’ but<br />

also ‘why does it work’. But to-date evaluati<strong>on</strong> has been dominated by overly positivistic paradigms.<br />

Instead, this study advocates focus groups as an interacti<strong>on</strong>ist approach that strives to incorporate<br />

various stakeholder interests and perspectives in order to achieve a c<strong>on</strong>sensus <strong>on</strong> these two<br />

questi<strong>on</strong>s. Focus groups c<strong>on</strong>sist of semi-structured questi<strong>on</strong> sessi<strong>on</strong>s in which a moderator promotes<br />

interacti<strong>on</strong> am<strong>on</strong>g a collecti<strong>on</strong> of participants that have been brought together to discuss and shed<br />

light <strong>on</strong> a particular topic, issue or c<strong>on</strong>cern. While focus groups are now <strong>on</strong>e of the most widely used<br />

research tools in the social sciences, their use to evaluate and refine design artefacts remains<br />

relatively new to the IS field and, in additi<strong>on</strong>, most of the literature that is available tends to be written<br />

by n<strong>on</strong>-designers. Focus groups do allow participants to express a range of opini<strong>on</strong>s that may not<br />

have been obvious if simply observing their behaviour. Focus groups are suited to design research in<br />

that they can offer designers a flexible range of techniques that can provide a c<strong>on</strong>sistent and<br />

encompassing research method supporting the full design process, including the determinati<strong>on</strong> of<br />

user needs (at the pre-c<strong>on</strong>cept stage), evaluati<strong>on</strong> of prototypes (during the design stage), and finally<br />

the testing of the final soluti<strong>on</strong>s (at the post-design stage). But the difficulty in facilitating and<br />

moderating focus groups should not be underestimated and planning can take weeks and even<br />

m<strong>on</strong>ths of effort. To fill this gap, the paper provided steps that should be followed when planning and<br />

organising focus groups as well as those steps that should be followed during and after a sessi<strong>on</strong>.<br />

References<br />

Avgerou, C. 1995. Evaluating Informati<strong>on</strong> Systems by C<strong>on</strong>sultati<strong>on</strong> and Negotiati<strong>on</strong>. Internati<strong>on</strong>al Journal of<br />

Informati<strong>on</strong> <strong>Management</strong> 15 (6):427-436.<br />

Barrow, P. D. M., and P. J. Mayhew. 2000. Investigating Principles of Stakeholder Evaluati<strong>on</strong> in a Modern IS<br />

Development Approach. Journal of Systems and Software 52 (2-3):95-103.<br />

Bruseberg, A., and D. McD<strong>on</strong>agh-Philp. 2002. Focus Groups to Support the Industrial/Product Designer: A<br />

Review Based <strong>on</strong> Current Literature and Designers' Feedback. Applied Erg<strong>on</strong>omics 33 (1):27-38.<br />

Cleven, A., P. Gubler, and K. M. Havner. 2009. Design Alternatives for the Evaluati<strong>on</strong> of Design Science<br />

Research Artifacts. In The 4th Internati<strong>on</strong>al <str<strong>on</strong>g>C<strong>on</strong>ference</str<strong>on</strong>g> <strong>on</strong> Design Science Research in Informati<strong>on</strong><br />

Systems and Technology (DESRIST). Malvern, PA, USA., 19.<br />

256


Paidi O’Raghallaigh, David Samm<strong>on</strong> and Ciaran Murphy<br />

Cr<strong>on</strong>holm, S., and G. Goldkuhl. 2003. Strategies for Informati<strong>on</strong> Systems Evaluati<strong>on</strong> - Six Generic Types.<br />

Electr<strong>on</strong>ic Journal of Informati<strong>on</strong> Systems Evaluati<strong>on</strong> 6 (2):65-74.<br />

Gibbs, A. 1997. Focus Groups. Social research update 19 (8).<br />

Gibs<strong>on</strong>, M., and D. Arnott. 2007. The Use of Focus Groups in Design Science Research. In The 18th<br />

Australasian <str<strong>on</strong>g>C<strong>on</strong>ference</str<strong>on</strong>g> <strong>on</strong> Informati<strong>on</strong> Systems (ACIS), edited by M. T. e. al. The University of Southern<br />

Queensland, Toowoomba, Australia, 327- 337.<br />

Goldkuhl, G., and M. Lind. 2010. A Multi-Grounded Design Research Process. In The 5th Internati<strong>on</strong>al<br />

<str<strong>on</strong>g>C<strong>on</strong>ference</str<strong>on</strong>g> I<strong>on</strong> Design Science Research in Informati<strong>on</strong> Systems and Technology (DESRIST) edited by R.<br />

Winter, J. L. Zhao and S. Aier. St.Gallen, Switzerland: Berlin: Springer, 45-60.<br />

Guba, E. G., and Y. S. Lincoln. 1989. Fourth Generati<strong>on</strong> Evaluati<strong>on</strong>. California, USA: Sage Publicati<strong>on</strong>s, Inc.<br />

Hansen, K., and R. S. Hansen. 2006. Using an Asynchr<strong>on</strong>ous Discussi<strong>on</strong> Board for Online Focus Groups: A<br />

Protocol and Less<strong>on</strong>s Learned. In The 2006 College Teaching & Learning <str<strong>on</strong>g>C<strong>on</strong>ference</str<strong>on</strong>g>. Orlando, FL, USA 1-<br />

8.<br />

Hevner, A. R. 2007. A Three Cycle View of Design Science Research. Scandinavian Journal of Informati<strong>on</strong><br />

Systems 19 (2):87-92.<br />

Hevner, A. R., S. T. March, J. Park, and S. Ram. 2004. Design Science in Informati<strong>on</strong> Systems Research. Mis<br />

Quarterly 28 (1):75-105.<br />

Iivari, J. 1988. Assessing IS Design Methodologies as Methods of IS Assessment. In Informati<strong>on</strong> Systems<br />

Assessment: Issues and Challenges, edited by N. Björn-Andersen and G. B. Davis. Amsterdam, The<br />

Netherlands: North-Holland, 59-78.<br />

———. 2007. A Paradigmatic Analysis of Informati<strong>on</strong> Systems As a Design Science. Scandinavian Journal of<br />

Informati<strong>on</strong> Systems 19 (2):39-64.<br />

Kitzinger, J. 1995. Qualitative Research: Introducing Focus Groups. British Medical Journal 311 (7000):299-302.<br />

K<strong>on</strong>tio, J., L. Lehtola, and J. Bragge. 2004. Using the Focus Group Method in Software Engineering: Obtaining<br />

Practiti<strong>on</strong>er and User Experiences. In The Internati<strong>on</strong>al Symposium <strong>on</strong> Empirical Software Engineering<br />

(ISESE). Red<strong>on</strong>do Beach, USA.<br />

Krueger, R. A., and M. A. Casey. 2000. Focus Groups: A Practical Guide for Applied Research. Thousand Oaks,<br />

CA., USA: Sage Publicati<strong>on</strong>s, Inc.<br />

March, S. T., and G. F. Smith. 1995. Design and Natural Science Research <strong>on</strong> Informati<strong>on</strong> Technology. Decisi<strong>on</strong><br />

Support Systems 15 (4):251-266.<br />

Mark, M. M. 1999. Toward an Integrative Framework for Evaluati<strong>on</strong> Practice. American Journal of Evaluati<strong>on</strong><br />

20:177-198.<br />

Markus, M. L., A. Majchrzak, and L. Gasser. 2002. A Design Theory for Systems That Support Emergent<br />

Knowledge Processes. Mis Quarterly 26 (3):179-212.<br />

Mazza, R., and A. Berre. 2007. Focus Group Methodology for Evaluating Informati<strong>on</strong> Visualizati<strong>on</strong> Techniques<br />

and Tools. In 11th Internati<strong>on</strong>al <str<strong>on</strong>g>C<strong>on</strong>ference</str<strong>on</strong>g> of Informati<strong>on</strong> Visualizati<strong>on</strong>. Zürich, Switzerland, 74-80.<br />

McD<strong>on</strong>agh-Philp, D., H. G. Dent<strong>on</strong>, and A. Bruseberg. 2000. The Use and Evaluati<strong>on</strong> of Focus Group Technique:<br />

The Undergraduate Industrial Designer Experience. Journal of the Nati<strong>on</strong>al Associati<strong>on</strong> for Design<br />

Educati<strong>on</strong> 8:17-26.<br />

Niehaves, B. 2007. On Epistemological Diversity in Design Science: New Vistas for a Design-Oriented IS<br />

Research? In The 28th Internati<strong>on</strong>al <str<strong>on</strong>g>C<strong>on</strong>ference</str<strong>on</strong>g> <strong>on</strong> Informati<strong>on</strong> Systems (ICIS). M<strong>on</strong>treal, Canada 133.<br />

Nielsen, J. 1997. The Use and Misuse of Focus Groups. Software, IEEE 14 (1):94-95.<br />

Peffers, K., T. Tuunanen, M. A. Rothenberger, and S. Chatterjee. 2007. A Design Science Research<br />

Methodology for Informati<strong>on</strong> Systems Research. Journal of <strong>Management</strong> Informati<strong>on</strong> Systems 24 (3):45-77.<br />

Powell, R. A., and H. M. Single. 1996. Focus Groups. Internati<strong>on</strong>al Journal for Quality in Health Care 8 (5):499.<br />

Pries-Heje, J., R. Baskerville, and J. R. Venable. 2008. Strategies for Design Science Research Evaluati<strong>on</strong>. In<br />

The 1<str<strong>on</strong>g>6th</str<strong>on</strong>g> <str<strong>on</strong>g>European</str<strong>on</strong>g> <str<strong>on</strong>g>C<strong>on</strong>ference</str<strong>on</strong>g> <strong>on</strong> Informati<strong>on</strong> Systems (ECIS) Galway, Ireland.<br />

Rezabek, R. J. 2000. Online Focus Groups: Electr<strong>on</strong>ic Discussi<strong>on</strong>s for Research. Forum Qualitative<br />

Sozialforschung/Forum: Qualitative Social Research. Available from http://www.qualitative-research.net/fqstexte/1-00/1-00rezabek-e.htm<br />

1.<br />

Serafeimidis, V., and S. Smiths<strong>on</strong>. 2003. Informati<strong>on</strong> Systems Evaluati<strong>on</strong> as an Organizati<strong>on</strong>al Instituti<strong>on</strong> -<br />

Experience from a Case Study. Informati<strong>on</strong> Systems Journal 13 (3):251-274.<br />

Smiths<strong>on</strong>, S., and R. Hirschheim. 1998. Analysing Informati<strong>on</strong> Systems Evaluati<strong>on</strong>: Another Look at an Old<br />

Problem. <str<strong>on</strong>g>European</str<strong>on</strong>g> Journal of Informati<strong>on</strong> Systems 7 (3):158-174.<br />

Sym<strong>on</strong>s, V. J. 1991. A Review of Informati<strong>on</strong> Systems Evaluati<strong>on</strong>: C<strong>on</strong>tent, C<strong>on</strong>text and Process. <str<strong>on</strong>g>European</str<strong>on</strong>g><br />

Journal of Informati<strong>on</strong> Systems 1 (3):205-212.<br />

Tremblay, M. C., A. R. Hevner, and D. J. Berndt. 2010. Focus Groups for Artifact Refinement and Evaluati<strong>on</strong> in<br />

Design Research. Communicati<strong>on</strong>s of the associati<strong>on</strong> for Informati<strong>on</strong> Systems 26 (1):27.<br />

Venable, J. 2010. Design Science Research Post Hevner et al.: Criteria, Standards, Guidelines, and<br />

Expectati<strong>on</strong>s. In The 5th Internati<strong>on</strong>al <str<strong>on</strong>g>C<strong>on</strong>ference</str<strong>on</strong>g> <strong>on</strong> Design Science in Informati<strong>on</strong> Systems and<br />

Technology (DESRIST). St. Gallen, Switzerland, 109-123.<br />

Walls, J. G., G. R. Widmeyer, and O. A. El Sawy. 1992. Building an Informati<strong>on</strong> System Design Theory for<br />

Vigilant EIS. Informati<strong>on</strong> Systems Research 3 (1):36-59.<br />

257


Realizing the Business Value of Service-Oriented<br />

Architecture: The C<strong>on</strong>structi<strong>on</strong> of a Theoretical Framework<br />

R<strong>on</strong>an O’Sullivan, Tom Butler and Philip O’Reilly<br />

University College Cork, Ireland<br />

r<strong>on</strong>an.osullivan@umail.ucc.ie<br />

TButler@afis.ucc.ie<br />

Philip.OReilly@ucc.ie<br />

Abstract: Service-oriented computing (SOC) has emerged over the past decade as an alternative and powerful<br />

approach to applicati<strong>on</strong> development and has sparked an increasing shift from inflexible proprietary software to<br />

more open service-oriented computing envir<strong>on</strong>ments. These service-oriented envir<strong>on</strong>ments focus <strong>on</strong> harnessing<br />

the power of the Internet and delivering business functi<strong>on</strong>ality through services. Organizati<strong>on</strong>s in many industries<br />

have turned to service-oriented computing envir<strong>on</strong>ments through the adopti<strong>on</strong> of service-oriented architecture<br />

(SOA). Services are the fundamental elements of SOA and are based <strong>on</strong> Internet standards and represent<br />

specific business functi<strong>on</strong>s. SOA is transporting organizati<strong>on</strong>s from the old world of inflexible and expensive<br />

traditi<strong>on</strong>al IT architecture to a brave new world where applicati<strong>on</strong>s are provided in the form of standardized<br />

services. Despite the increasing adopti<strong>on</strong> of SOA within academia and practice, an analysis of the extant<br />

literature by this study reveals a clear lack of research <strong>on</strong> the business perspective of SOA and in particular <strong>on</strong><br />

the business value of SOA. Indeed, the business value of IT (BVIT) research area - a fundamental area of<br />

research within the IS discipline - is c<strong>on</strong>sidered by many as being under-researched and in need of an expanded<br />

research agenda. This study c<strong>on</strong>structs a theoretical framework and develops a set of propositi<strong>on</strong>s and<br />

hypotheses to investigate how the business value of SOA is realized. It illustrates that the combinati<strong>on</strong> of a SOA<br />

implementati<strong>on</strong> and complementary resources enable the creati<strong>on</strong> of SOA-enabled resources via multiple<br />

enablers. These SOA-enabled resources produce emergent SOA capabilities which realize business value at the<br />

process level and at the level of the organizati<strong>on</strong>.<br />

Keywords: service-oriented architecture, business value of IT, business value of SOA, dynamic capabilities,<br />

complementary resources<br />

1. Introducti<strong>on</strong> and rati<strong>on</strong>ale<br />

Over the past decade, a fundamental shift in the way applicati<strong>on</strong>s are developed and deployed has<br />

originated with the emergence of SOA which is at the heart of the e-business revoluti<strong>on</strong> (Chen 2008)<br />

and is hailed by Maurizio et al (2008) as a key disruptive technology in the pursuit of competitive<br />

advantage from IT. In a 2010 survey, TechTarget and Forrester Research (2010) describe SOA as<br />

being ‘entrenched’ in today’s business world and indicates that almost half of resp<strong>on</strong>dents are<br />

working in organizati<strong>on</strong>s where SOA projects are underway.<br />

Despite the increasing rate of SOA adopti<strong>on</strong>, there is a significant lack of research <strong>on</strong> the business<br />

perspective of SOA and <strong>on</strong> the business value of SOA. A heavy focus <strong>on</strong> the technological<br />

perspective of SOA has rendered the business perspective under-researched (Biemborn et al, 2008;<br />

Luthria and Rabhi 2008; Luthria et al, 2007). Luthria and Rabhi (2008) capture the essence of the gap<br />

by stating that the technological perspective has been appropriately addressed but the business or<br />

practical use of SOA has not. Most organizati<strong>on</strong>s who have adopted SOA are using it as a<br />

technological initiative rather than as a business transformati<strong>on</strong> tool (Merrifield et al, 2008) and as a<br />

result, most organizati<strong>on</strong>s that adopt SOA do not fully understand the business potential (Luthria and<br />

Rabhi 2008).<br />

Scratching deeper under the surface, a significant body of researchers lament the lack of research <strong>on</strong><br />

the business value of SOA and call for it to be investigated (e.g. Luthria and Rabhi 2008; Viering et al,<br />

2009; Biemborn et al, 2008; K<strong>on</strong>togiannis et al, 2008). Viering et al (2009) call for theories to be<br />

applied and extended to understand how SOA improves an organizati<strong>on</strong>’s capabilities to realize<br />

business value, while both K<strong>on</strong>togiannis et al (2008) and Biemborn et al (2008) call for the<br />

development of a comprehensive framework for understanding the business value of SOA.<br />

This study addresses these gaps by proposing a theoretical framework to understand how the<br />

business value of SOA is realized. The key objective of the study is to embark <strong>on</strong> a theoretical<br />

trajectory and leverage previous work in the BVIT research area in order to c<strong>on</strong>struct a theoretical<br />

framework for understanding how SOA realizes business value.<br />

258


R<strong>on</strong>an O’Sullivan, Tom Butler and Philip O’Reilly<br />

This study begins with an analysis of the SOA and BVIT literature and develops thorough<br />

c<strong>on</strong>ceptualizati<strong>on</strong>s of both research areas. The next secti<strong>on</strong> is devoted to the c<strong>on</strong>structi<strong>on</strong> of a<br />

theoretical framework and the development of a set of hypotheses to explain how SOA is leveraged to<br />

realize business value. In the first part, the theoretical perspective which comprises the resourcebased<br />

view of the firm (RBV), dynamic capabilities and Net-Enabled Business Innovati<strong>on</strong> Cycle<br />

(NEBIC) theory is explained and justified. The sec<strong>on</strong>d part of the secti<strong>on</strong> presents the framework<br />

which focuses <strong>on</strong> the creati<strong>on</strong> of SOA-enabled resources from the combinati<strong>on</strong> of a SOA<br />

implementati<strong>on</strong> and complementary resources, and the emergence of SOA capabilities. Finally, a<br />

c<strong>on</strong>clusi<strong>on</strong> is presented which also provides the findings from a preliminary analysis and an agenda<br />

for future research.<br />

2. An analysis of the SOA literature<br />

The recent rise in popularity of SOA has fuelled the explosi<strong>on</strong> of academic and practiti<strong>on</strong>er literature.<br />

Various c<strong>on</strong>ceptualizati<strong>on</strong>s of SOA have been produced and key similarities exist throughout. First,<br />

there is c<strong>on</strong>sensus that SOA represents an IT architecture that presents an alternative approach to<br />

applicati<strong>on</strong> development through the utilizati<strong>on</strong> and interacti<strong>on</strong> of standardized services (e.g.<br />

Merrifield et al, 2008; Baskerville et al, 2005; Haki and Forte 2010). For instance, Merrifield et al<br />

(2008) and Baskerville et al (2005) describe it as a new way of designing and deploying software that<br />

supports business activities, while Haki and Forte (2010) label it “a new way of developing systems<br />

that promotes a shift from writing software to assembling and integrating services” (Haki and Forte<br />

2010).<br />

The sec<strong>on</strong>d similarity c<strong>on</strong>tends that services, which play an integral role in service-oriented<br />

architecture, are based up<strong>on</strong> business functi<strong>on</strong>s. SOA allows business functi<strong>on</strong>s to be developed as<br />

modular services which can be called up<strong>on</strong> and reused when they are needed within business<br />

processes (Maurizio et al, 2008). This provides organizati<strong>on</strong>s with the opportunity for firms to<br />

fundamentally redesign their operati<strong>on</strong>s (Merrifield et al, 2008) and develop new functi<strong>on</strong>ality to<br />

satisfy the diverse requirements of potential service c<strong>on</strong>sumers (Chen 2008).<br />

Thirdly, the SOA literature highlights the key characteristics of SOA (e.g. Hau et al, 2008, Merrifield et<br />

al, 2008). Table 1 describes each characteristic.<br />

Table 1: The characteristics of SOA<br />

Characteristics Descripti<strong>on</strong> References<br />

SOA is based <strong>on</strong><br />

Internet<br />

Standards<br />

Service<br />

integrati<strong>on</strong><br />

Service<br />

modularity<br />

Service<br />

reusability<br />

C<strong>on</strong>tractorientati<strong>on</strong><br />

SOA is a net-enabling technology as services are<br />

built up<strong>on</strong> Internet standards such as HTTP,<br />

SOAP, WSDL, and UDDI which allow services to<br />

be easily integrated and rec<strong>on</strong>figured.<br />

The standardized services which comprise SOA<br />

facilitate platform- and language-independent<br />

integrati<strong>on</strong> with each other.<br />

Services are modular (or aut<strong>on</strong>omous) and can be<br />

combined and rearranged in a flexible and timely<br />

manner without any unpredictable effect <strong>on</strong> other<br />

services.<br />

SOA allows and encourages service c<strong>on</strong>sumers to<br />

reuse services in different business c<strong>on</strong>texts.<br />

Service c<strong>on</strong>tracts publicly establish the terms of<br />

engagement between services and the<br />

collaborating partners such as specifying the<br />

functi<strong>on</strong>ality of a particular service.<br />

Baskerville et al (2005),<br />

Hau et al (2008),<br />

Merrifield et al (2008)<br />

Baskerville et al (2005),<br />

Hagel and Seely Brown (2001),<br />

Papazoglou et al (2007)<br />

Hau et al (2008),<br />

Maurizio et al (2008),<br />

Chen (2008)<br />

Baskerville et al (2005),<br />

Hau et al (2008),<br />

Merrifield et al (2008)<br />

Hau et al. (2008),<br />

Maurizio et al. (2008),<br />

Chen (2008)<br />

From the SOA descripti<strong>on</strong>s and characteristics in Table 1, this study defines SOA as an architectural<br />

style that offers an alternative approach to applicati<strong>on</strong> development and delivery through the<br />

utilizati<strong>on</strong> and combinati<strong>on</strong> of standardized, modular and reusable services. These services represent<br />

specific business functi<strong>on</strong>s which are the primary focus for service-oriented development.<br />

The benefits of SOA have been a subject of interest for SOA authors (e.g. Merrifield et al, 2008;<br />

Baskerville et al, 2005). Table 2 describes each of the key business benefits of SOA.<br />

259


Table 2: The business benefits of SOA<br />

Business<br />

Benefits<br />

Reduced IT<br />

Costs<br />

Enhanced<br />

Innovati<strong>on</strong><br />

Increased<br />

Flexibility<br />

R<strong>on</strong>an O’Sullivan, Tom Butler and Philip O’Reilly<br />

Descripti<strong>on</strong> References<br />

Reduced Integrati<strong>on</strong> Costs: SOA provides the ability to easily<br />

and cost-effective and ubiquitously share software modules<br />

with internal and external clients.<br />

Reduced Applicati<strong>on</strong> Development Costs: SOA reduces<br />

development costs through the reuse of modular services in<br />

separate applicati<strong>on</strong>s and in different c<strong>on</strong>texts.<br />

SOA enables new, innovative service-oriented business<br />

soluti<strong>on</strong>s to be developed through the combinati<strong>on</strong> of existing<br />

and different services and the opportunities for new<br />

functi<strong>on</strong>ality.<br />

Service-oriented applicati<strong>on</strong>s can be quickly and costeffectively<br />

assembled and reassembled to resp<strong>on</strong>d to changing<br />

market c<strong>on</strong>diti<strong>on</strong>s or demand.<br />

3. An analysis of the BVIT literature<br />

Merrifield et al (2008),<br />

Baskerville et al<br />

(2005),<br />

Maurizio et al (2008),<br />

Papazoglou et al<br />

(2007),<br />

Hagel and Seely<br />

Brown (2001),<br />

Merrifield et al (2008)<br />

Hau et al (2008),<br />

Bieberstein et al<br />

(2005), Merrifield et al<br />

2008)<br />

Since the late 1980s a significant amount of literature has explored the c<strong>on</strong>tributi<strong>on</strong> of IT to business<br />

performance, so-much-so that by the late mid-to-late 2000s the BVIT had solidified its place at the<br />

core of the IS research field (Kohli and Grover 2008; Nevo and Wade 2010; Agarwal and Lucas<br />

2005). The importance of the research area is emphasized by Barua et al (1995) who c<strong>on</strong>tend that<br />

“measuring the ec<strong>on</strong>omic c<strong>on</strong>tributi<strong>on</strong> of IT investments is a key activity that can shape the very<br />

nature of business through its influence <strong>on</strong> corporate strategies and future investments in technology”<br />

(Barua et al, 1995). Nevo and Wade (2010) agree with this sentiment as they refer to the integral and<br />

strategic role of IT in modern organizati<strong>on</strong>s. In general, the importance of IT for business is brought<br />

across by the c<strong>on</strong>sensus am<strong>on</strong>g authors and practiti<strong>on</strong>ers that IT improves business performance and<br />

creates business value (e.g. Kohli and Grover 2008; Brynjolfss<strong>on</strong> and Hitt 2000).<br />

The BVIT literature highlights several characteristics that define the BVIT research area. Table 3<br />

indicates these characteristics and provides a descripti<strong>on</strong> for each.<br />

Table 3: Characteristics of the BVIT research area<br />

Characteristic Explanati<strong>on</strong><br />

The area is under- The IS field is not doing enough to explain how organizati<strong>on</strong>s are realizing<br />

researched<br />

The dominati<strong>on</strong> of<br />

Firm-level research<br />

The difficulty of<br />

measuring BVIT<br />

The importance of<br />

complementary<br />

resources<br />

The dominant<br />

applicati<strong>on</strong> of the RBV<br />

in the research area<br />

business value through IT (Kohli and Grover 2008; Nevo and Wade 2010).<br />

Firm-level research is c<strong>on</strong>sidered to be the most effective way of dem<strong>on</strong>strating a<br />

positive relati<strong>on</strong>ship between IT investments and organizati<strong>on</strong>al performance<br />

(e.g. Brynjolfss<strong>on</strong> and Hitt 2000).<br />

BVIT measurement represents an ever-present problem to the BVIT research<br />

area as it is very difficult to attribute value to IT (Melville et al, 2004; Barua et al,<br />

2004; Brynolfss<strong>on</strong> and Hitt 2000). Research has typically focused <strong>on</strong> traditi<strong>on</strong>al<br />

accounting measures however, these measures are now c<strong>on</strong>sidered inadequate<br />

in isolati<strong>on</strong> as indirect or intangible factors are difficult to measure through<br />

traditi<strong>on</strong>al accounting techniques (Kohli and Grover 2008; Melville et al, 2004).<br />

Complementary resources are central to the realizati<strong>on</strong> of BVIT as the IT asset<br />

cannot be c<strong>on</strong>sidered in isolati<strong>on</strong> in the pursuit of improved business<br />

performance (Melville et al, 2004; Nevo and Wade 2010; Barua et al, 2004).<br />

The importance of complementary resources to BVIT research is a key factor for<br />

the widespread applicati<strong>on</strong> of the RBV in the research area (Barua et al, 2004;<br />

Melville et al, 2004; Kohli and Grover 2008).<br />

This study broadly c<strong>on</strong>ceptualizes BVIT as the c<strong>on</strong>tributi<strong>on</strong> of IT to business performance and adopts<br />

the BVIT definiti<strong>on</strong> of Melville et al (2004): “the organizati<strong>on</strong>al performance impacts of informati<strong>on</strong><br />

technology at both the intermediate process level and the organizati<strong>on</strong> wide level, and comprising<br />

both efficiency impacts and competitive impacts” (Melville et al, 2004).<br />

The next secti<strong>on</strong> is devoted to the c<strong>on</strong>structi<strong>on</strong> of a theoretical framework in order to understand how<br />

SOA realizes business value<br />

260


R<strong>on</strong>an O’Sullivan, Tom Butler and Philip O’Reilly<br />

4. Toward a theoretical framework for understanding the business value of<br />

SOA<br />

This secti<strong>on</strong> begins with a detailed rati<strong>on</strong>ale for the theoretical perspective that encompasses the<br />

RBV, dynamic capabilities, and NEBIC theory. The rati<strong>on</strong>ale for each is based up<strong>on</strong> their relati<strong>on</strong>ships<br />

with each other, their c<strong>on</strong>necti<strong>on</strong>s to the c<strong>on</strong>cept of SOA, and the c<strong>on</strong>tenti<strong>on</strong>s that SOA is a netenabling<br />

technology and net-enablement is a dynamic capability. A theoretical framework is then<br />

presented which applies tenets from each of the three theories and builds primarily up<strong>on</strong> the work of<br />

Nevo and Wade (2010) who c<strong>on</strong>tend that the combinati<strong>on</strong> of an IT asset and complementary<br />

resources can ultimately create business value.<br />

4.1 The rati<strong>on</strong>ale for the theoretical perspective<br />

The rati<strong>on</strong>ale for this study’s theoretical perspective which encompasses the resource-based view of<br />

the firm, dynamic capabilities and Net-Enabled Business Innovati<strong>on</strong> Cycle (NEBIC) theory is<br />

explained below.<br />

4.1.1 The RBV<br />

The RBV is a theory that emphasizes distinctive resources are the basis for achieving and retaining<br />

competitive advantage (Barney 1991; Wernerfelt 1984) and is extremely useful in the BVIT research<br />

area (e.g. Melville et al, 2004; Kohli and Grover 2008). The RBV is c<strong>on</strong>sidered to be the basetheoretical<br />

perspective for the study for three key reas<strong>on</strong>s: first, SOA is c<strong>on</strong>sidered by many as being<br />

c<strong>on</strong>ceptually close to RBV as SOA sees the leveraging of existing applicati<strong>on</strong>s into services which<br />

can be rapidly recombined into new business soluti<strong>on</strong>s, while the RBV c<strong>on</strong>tends that competitive<br />

advantage can be attained through the combinati<strong>on</strong> of heterogeneous resources (e.g. Chen 2008;<br />

Choi and Ramamurthy 2011). Sec<strong>on</strong>d, this study asserts that the firm’s resources are of central<br />

importance as the combinati<strong>on</strong> of SOA and complementary resources create SOA-enabled resources<br />

which are proposed as sources of business value (see Figure 2). Finally, both the dynamic<br />

capabilities and NEBIC theory are inherently based <strong>on</strong> the RBV.<br />

4.1.2 Dynamic capabilities<br />

Dynamic capabilities provide the ability to integrate, rec<strong>on</strong>figure, add and dispose of internal and<br />

external resources, and enable firms to perform these acti<strong>on</strong>s in resp<strong>on</strong>se to c<strong>on</strong>tinuously changing<br />

business envir<strong>on</strong>ment c<strong>on</strong>diti<strong>on</strong>s (Eisenhardt and Martin 2000; Teece et al, 1997). The development<br />

of the dynamic capabilities approach stems from the key limitati<strong>on</strong>s of the RBV: the failure to: (a)<br />

explain how resources are developed, integrated, and released, and (b) explain how firms achieve<br />

competitive advantage in unstable or dynamic business envir<strong>on</strong>ments. Table 4 identifies and explains<br />

the different roles of dynamic capabilities in enabling firms to increase competitiveness. Examples of<br />

dynamic capabilities include net-enablement (Wheeler 2002) and product development routines<br />

(Eisenhardt and Martin 2000).<br />

Table 4: The roles of dynamic capabilities<br />

Role Descripti<strong>on</strong><br />

Integrati<strong>on</strong> The efficient and effective integrati<strong>on</strong> of internal and external resources, skills and<br />

activities allows firms to develop such dynamic capabilities which enable them to<br />

increase competitiveness in dynamic business envir<strong>on</strong>ments (Teece et al, 1997;<br />

Eisenhardt and Martin 2000).<br />

Learning Learning processes are vital to the dynamic capabilities approach as they enable tasks<br />

to be performed better and more quickly, create knowledge, and identify new<br />

opportunities (Teece et al, 1997; Eisenhardt and Martin 2000).<br />

Rec<strong>on</strong>figurati<strong>on</strong> The rec<strong>on</strong>figurati<strong>on</strong> of a firm’s resources and capabilities is vital in order to enable a firm<br />

to change their bundle of resources in order to adapt to change. (Teece et al, 1997;<br />

Eisenhardt and Martin 2000).<br />

The dynamic capabilities approach is regarded as the most critical theoretical perspective for two key<br />

reas<strong>on</strong>s: first, there is an c<strong>on</strong>sensus that SOA is heavily influencing and enabling the development of<br />

dynamic capabilities (e.g. Luthria et al, 2007; Choi and Ramamurthy 2011; Luthria and Rabhi 2008).<br />

For instance, Luthria et al (2007) describes SOA as “the technology infrastructure required to<br />

implement a firm’s dynamic capabilities” (Luthria et al, 2007). Sec<strong>on</strong>d, the central IT artifact, the SOA<br />

implementati<strong>on</strong>, is a net-enabling technology and net-enablement is declared by Wheeler (2002) as a<br />

261


R<strong>on</strong>an O’Sullivan, Tom Butler and Philip O’Reilly<br />

dynamic capability. It is for these reas<strong>on</strong>s that the dynamic capabilities approach can be c<strong>on</strong>sidered<br />

extremely suitable for studying SOA.<br />

4.1.3 NEBIC theory<br />

The explosi<strong>on</strong> of Internet technologies and in particular, net-enabled business transacti<strong>on</strong>s (e.g. Weill<br />

and Vitale 2001) has fuelled the development of NEBIC theory - an applied dynamic capabilities<br />

theory which integrates the fields of strategic management and IS research through the dynamic<br />

capability of net-enablement (Wheeler 2002). Net-enablement enables organizati<strong>on</strong>s to leverage<br />

pervasive digital networks (i.e. the Internet) in order to rec<strong>on</strong>figure their resources and exploit<br />

business opportunities in dynamic business envir<strong>on</strong>ments. NEBIC theory c<strong>on</strong>tends that enabling<br />

technologies create or reveal ec<strong>on</strong>omic opportunities which can be transformed through business<br />

innovati<strong>on</strong> into customer value (Wheeler 2002). Figure 1 dem<strong>on</strong>strates the thesis of NEBIC theory.<br />

Figure 1: Wheeler’s (2002) NEBIC theory<br />

NEBIC theory is leveraged within the theoretical perspective for three key reas<strong>on</strong>s. First, NEBIC is<br />

specifically developed for the IS research field, and in particular is suited to the BVIT research area as<br />

the objective is creating value. Sec<strong>on</strong>d, net-enablement is decreed as a dynamic capability, and third,<br />

SOA is a net-enabling technology as <strong>on</strong>e of its key characteristics is that it is based <strong>on</strong> Internet<br />

standards (Baskerville et al, 2005; Merrifield et al, 2008).<br />

4.1.4 Applying the theoretical perspective to the theoretical framework<br />

The theoretical perspective is applied to the framework in the following way. The RBV stresses the<br />

importance of resources to firm-competitiveness and this is recognized and applied to the framework<br />

as resources are vital in associati<strong>on</strong> with IT assets and are key sources for realizing BVIT. NEBIC<br />

theory and dynamic capabilities are applied in c<strong>on</strong>juncti<strong>on</strong> with each other as NEBIC Theory c<strong>on</strong>tends<br />

that net-enablement is a dynamic capability and because SOA is deemed net-enabling, the<br />

framework applies the dynamic capabilities approach through its key roles of integrati<strong>on</strong>, learning and<br />

rec<strong>on</strong>figurati<strong>on</strong> in the development of specific resources.<br />

4.2 A theoretical framework for understanding the business value of SOA<br />

Based <strong>on</strong> a literature review of SOA and BVIT and the theoretical perspectives explained above, a<br />

theoretical framework has been c<strong>on</strong>structed as illustrated in Figure 2. In this framework, the<br />

implementati<strong>on</strong> of SOA and complementary resources are combined to create SOA-enabled<br />

resources via multiple enablers. Complementary resources c<strong>on</strong>stitute organizati<strong>on</strong>al and<br />

envir<strong>on</strong>mental resources which are applied in c<strong>on</strong>juncti<strong>on</strong> with an IT asset in order to achieve<br />

organizati<strong>on</strong>al performance impacts (Melville et al, 2004; Barua et al, 2004; Nevo and Wade 2010).<br />

262


R<strong>on</strong>an O’Sullivan, Tom Butler and Philip O’Reilly<br />

Examples of complementary resources and factors include: training (Kohli and Grover 2008),<br />

business processes (Melville et al, 2004), and IT management skills (Mata et al, 1995). An example of<br />

a SOA-enabled resource is the partnership between SOA and BPM (Business Process <strong>Management</strong>)<br />

with SOA providing the capabilities for manipulating standardized, modular and reusable services,<br />

and BPM providing the ability to optimize business processes in a service-oriented manner (Bajwa et<br />

al, 2009; Brahe, 2007).<br />

The framework then proposes that SOA-enabled resources produce emergent SOA capabilities which<br />

realize business value at the process level and at the level of the organizati<strong>on</strong>. Emergent SOA<br />

capabilities refer to outcomes or acti<strong>on</strong>s which are classed as either predictable or unpredictable and<br />

technical or strategic in nature (Nevo and Wade 2010). Examples of emergent SOA capabilities<br />

include: reduced applicati<strong>on</strong> development and maintenance time and costs, the reuse of service, and<br />

to rapidly and easily rec<strong>on</strong>figure informati<strong>on</strong> systems to adapt to change (e.g. Merrifield et al, 2008).<br />

This descripti<strong>on</strong> of the theoretical framework is based up<strong>on</strong> an analysis of the SOA literature and<br />

Nevo and Wade’s (2010) argument that “the full extent of IT assets’ business value may not become<br />

apparent until they are placed in a relati<strong>on</strong>ship with organizati<strong>on</strong>al resources and used to create ITenabled<br />

resources” (Nevo and Wade 2010).<br />

Figure 2: A theoretical framework for understanding the business value of SOA<br />

4.2.1 The creati<strong>on</strong> of SOA-enabled resources<br />

An IT asset cannot be c<strong>on</strong>sidered in isolati<strong>on</strong> but rather in combinati<strong>on</strong> with complementary resources<br />

in the pursuit of improved business performance (Brynjolfss<strong>on</strong> and Hitt 2000; Nevo and Wade 2010;<br />

Kohli and Grover 2008; Barua et al, 2004). Through the combinati<strong>on</strong> of technological, organizati<strong>on</strong>al<br />

and envir<strong>on</strong>mental resources, a set of capabilities can be developed to improve operati<strong>on</strong>al and<br />

financial performance (Barua et al, 2004). Similarly, Nevo and Wade (2010) claim IT assets can be<br />

combined with organizati<strong>on</strong>al resources to create synergistic IT-enabled resources and c<strong>on</strong>sequently,<br />

emergent capabilities in order to achieve and retain competitive advantage. This study c<strong>on</strong>tends that<br />

SOA-enabled resources are the result of the combinati<strong>on</strong> of the SOA implementati<strong>on</strong> and<br />

complementary organizati<strong>on</strong>al and envir<strong>on</strong>mental resources.<br />

In order to create IT-enabled resources, Nevo and Wade (2010) emphasize two specific enablers -<br />

compatibility and integrati<strong>on</strong>. Compatibility refers to the “ability of an organizati<strong>on</strong>al resource to apply<br />

an IT asset in its regular activities and routines” (Nevo and Wade 2010), and integrati<strong>on</strong> refers to<br />

“activities taken by the organizati<strong>on</strong>’s management to support, guide and assist the implementati<strong>on</strong> of<br />

the IT asset within the organizati<strong>on</strong>al resource” (Nevo and Wade 2010). This study c<strong>on</strong>tends that the<br />

key roles of dynamic capabilities (see Table 4) are potential enablers of SOA-enabled resources as<br />

SOA is a net-enabling technology and net-enablement is a dynamic capability. Similar to compatibility<br />

263


R<strong>on</strong>an O’Sullivan, Tom Butler and Philip O’Reilly<br />

and integrati<strong>on</strong>, these roles are made up of routines and activities and enable firms to increase<br />

competitiveness. Therefore, the following propositi<strong>on</strong> is developed:<br />

Propositi<strong>on</strong> 1 - The creati<strong>on</strong> of SOA-enabled resources through the combinati<strong>on</strong> of a SOA<br />

implementati<strong>on</strong> and complementary resources is enabled through the compatibility, integrati<strong>on</strong>,<br />

rec<strong>on</strong>figurati<strong>on</strong> of resources and learning.<br />

We refine Propositi<strong>on</strong> 1 by specifying four hypotheses as illustrated in Figure 2:<br />

H1a - The creati<strong>on</strong> of SOA-enabled resources through the combinati<strong>on</strong> of a SOA implementati<strong>on</strong> and<br />

complementary resources is enabled through the compatibility of resources.<br />

H1b - The creati<strong>on</strong> of SOA-enabled resources through the combinati<strong>on</strong> of a SOA implementati<strong>on</strong> and<br />

complementary resources is enabled through the integrati<strong>on</strong> of resources.<br />

H1c - The creati<strong>on</strong> of SOA-enabled resources through the combinati<strong>on</strong> of a SOA implementati<strong>on</strong> and<br />

complementary resources is enabled through the rec<strong>on</strong>figurati<strong>on</strong> of resources.<br />

H1d - The creati<strong>on</strong> of SOA-enabled resources through the combinati<strong>on</strong> of a SOA implementati<strong>on</strong> and<br />

complementary resources is enabled through learning.<br />

4.2.2 The realizati<strong>on</strong> of business value of SOA through emergent SOA capabilities<br />

The emergence of capabilities from the relati<strong>on</strong>ship between complementary resources and IT are<br />

regarded as sources of business value (Barua et al, 2004; Nevo and Wade 2010). According to Nevo<br />

and Wade (2010), specific capabilities emerge from synergistic IT-enabled resources which “can be<br />

used to attain and sustain competitive advantage”, while Barua et al, (2004) claim these emergent<br />

capabilities have the ability to realize business value. Furthermore, the definiti<strong>on</strong> of BVIT provided by<br />

Melville et al, (2004) c<strong>on</strong>tends that the realizati<strong>on</strong> of business value occurs at the process-level and at<br />

the level of the organizati<strong>on</strong>. Therefore, this study proposes the following propositi<strong>on</strong>s:<br />

Propositi<strong>on</strong> 2 - SOA-enabled resources produce SOA capabilities which directly realize business<br />

value.<br />

Propositi<strong>on</strong> 3 - The SOA capabilities which emerge from the SOA-enabled resources realize business<br />

value at the process level and at the level of the organizati<strong>on</strong>.<br />

We refine Propositi<strong>on</strong> 3 by specifying two hypotheses as illustrated in Figure 2:<br />

H3a - The SOA capabilities which emerge from the SOA-enabled resources realize business value at<br />

the process level.<br />

H3b - The SOA capabilities which emerge from the SOA-enabled resources realize business value at<br />

the level of the organizati<strong>on</strong>.<br />

5. C<strong>on</strong>clusi<strong>on</strong> and future research<br />

Despite a significant body of research <strong>on</strong> service-oriented computing and SOA, there is a dearth of<br />

research <strong>on</strong> the business perspective of SOA and particularly the business value of SOA. The<br />

objective of the study is to address this gap in the literature by c<strong>on</strong>structing a theoretical framework for<br />

understanding for the business value of SOA<br />

This paper makes a number of theoretical c<strong>on</strong>tributi<strong>on</strong>s to the extant literature. Firstly, it presents a<br />

theoretical framework for understanding the business value of SOA. In developing this framework,<br />

RBV, dynamic capabilities and NEBIC theory is used in c<strong>on</strong>juncti<strong>on</strong> with previous work that focuses<br />

<strong>on</strong> the combinati<strong>on</strong> of the IT asset and complementary resources (e.g. Nevo and Wade 2010; Barua<br />

et al, 2006). The result is a framework which suggests that the combinati<strong>on</strong> of a SOA implementati<strong>on</strong><br />

and complementary resources enable the creati<strong>on</strong> of SOA-enabled resources via multiple enablers.<br />

Seen as net-enablement is a dynamic capability, the framework extends Nevo and Wade’s (2010)<br />

enablers from integrati<strong>on</strong> and compatibility to include rec<strong>on</strong>figurati<strong>on</strong> and learning (Teece et al, 1997;<br />

Eisenhardt and Martin 2000). These SOA-enabled resources produce emergent SOA capabilities<br />

264


R<strong>on</strong>an O’Sullivan, Tom Butler and Philip O’Reilly<br />

which realize business value at the process level and at the level of the organizati<strong>on</strong>. Furthermore,<br />

the theoretical value of the paper is that it applies a different type of resource-based theoretical<br />

perspective to the work carried out primarily by Nevo and Wade (2010). The RBV is heavily<br />

incorporated into Nevo and Wade’s (2010) work however, the theoretical perspective incorporated by<br />

this study is expanded and is based <strong>on</strong> the RBV, dynamic capabilities and NEBIC theory. Dynamic<br />

capabilities is most prominent as the framework transforms its key roles (see Table 4) into enablers<br />

for the creati<strong>on</strong> of SOA-enabled resources which produce emergent SOA capabilities that are direct<br />

sources of business value.<br />

This paper also has great potential value for practiti<strong>on</strong>ers. Through utilizati<strong>on</strong> of the theoretical<br />

framework, it potentially offers an insight into how organizati<strong>on</strong>s can optimally manage their SOA<br />

implementati<strong>on</strong> in order to maximize value from their SOA. It can also be used to inform practiti<strong>on</strong>ers<br />

of the nature of the business value which arises as a result of a SOA implementati<strong>on</strong>.<br />

The researchers are now calling <strong>on</strong> further research to empirically test and validate the presented<br />

model. One potential approach would c<strong>on</strong>sist of a two-stage multi-method approach, comprising of a<br />

combinati<strong>on</strong> of case studies and an organizati<strong>on</strong> survey of SOA practices.<br />

References<br />

Agarwal, R. and Lucas, H.C. (2005) "The Informati<strong>on</strong> Systems Identity Crisis: Focusing <strong>on</strong> High-Visibility and<br />

High-Impact Research", MIS Quarterly, Vol. 29, No. 3, September, pp 381-398.<br />

Bajwa, I.S. et al (2009) “SOA and BPM Partnership: A Paradigm for Dynamic and Flexible Process and I.T.<br />

<strong>Management</strong>”, Internati<strong>on</strong>al Journal of Human and Social Sciences Vol. 4, No. 7, pp 554-560.<br />

Barney, J.B. (1991) “Firm Resources and Sustained Competitive Advantage”, Journal of <strong>Management</strong>, Vol. 17,<br />

No. 1, March, pp 99-120.<br />

Barua, A., Kriebel, C. and Mukhopadhyay, T. (1995) "Informati<strong>on</strong> Technologies and Business Value: An Analytic<br />

and Empirical Investigati<strong>on</strong>", Informati<strong>on</strong> Systems Research, Vol 6, No. 1, March, pp 3-23.<br />

Barua, A. et al (2004) “An Empirical Investigati<strong>on</strong> of Net-Enabled Business Value”, MIS Quarterly, Vol. 28, No. 4,<br />

December, pp 585-620.<br />

Baskerville, R. et al (2005) “Extensible Architectures: The Strategic Value of Service-oriented Architecture in<br />

Banking”, Proceedings of the 13th <str<strong>on</strong>g>European</str<strong>on</strong>g> <str<strong>on</strong>g>C<strong>on</strong>ference</str<strong>on</strong>g> <strong>on</strong> Informati<strong>on</strong> Systems, Regensburg, Germany,<br />

Vol. 341, No 1-2, May, pp 106-111.<br />

Beimborn, D., Joachim, N. and Weitzel, T. (2008) "Drivers and Inhibitors of SOA Business Value -<br />

C<strong>on</strong>ceptualizing a Research Model", AMCIS 2008 Proceedings, Paper 227.<br />

Bieberstein, N. et al (2005) “Impact of Service-oriented Architecture <strong>on</strong> Enterprise Systems, Organizati<strong>on</strong>al<br />

Structures, and Individuals”, IBM Systems Journal, Vol. 44, No. 4, pp 691-708.<br />

Brahe, S. (2007) “BPM <strong>on</strong> Top of SOA: Experiences from the Financial Industry”, Proceedings of the 5th<br />

Internati<strong>on</strong>al <str<strong>on</strong>g>C<strong>on</strong>ference</str<strong>on</strong>g> <strong>on</strong> Business Process <strong>Management</strong>, Germany, pp 96-111.<br />

Brynjolfss<strong>on</strong>, E. and Hitt, L.M. (2000) "Bey<strong>on</strong>d Computati<strong>on</strong>: Informati<strong>on</strong> Technology, Organizati<strong>on</strong>al<br />

Transformati<strong>on</strong> and Business Performance”, Journal of Ec<strong>on</strong>omic Perspectives, Vol. 14, No. 4, Autumn, pp<br />

23-48.<br />

Chen, H.M. (2008) “Towards Service Engineering, Service Orientati<strong>on</strong> and Business-IT Alignment”, Proceedings<br />

of the 41st Hawaii Internati<strong>on</strong>al <str<strong>on</strong>g>C<strong>on</strong>ference</str<strong>on</strong>g> <strong>on</strong> System Sciences, USA, January.<br />

Choi, J. and Ramamurthy, K. (2011) “Service-Oriented Architecture and IT-Business Alignment”, Proceedings of<br />

the 2011 Internati<strong>on</strong>al <str<strong>on</strong>g>C<strong>on</strong>ference</str<strong>on</strong>g> <strong>on</strong> Industrial Engineering and Operati<strong>on</strong>s <strong>Management</strong>, Kuala Lumpur,<br />

Malaysia, January pp 480-485.<br />

Eisenhardt, K.M. and Martin, J.A. (2000) "Dynamic Capabilities: What Are They?", Strategic <strong>Management</strong><br />

Journal, Vol, 21, No. 10-11, October-November, pp 1105-1121.<br />

Hagel III, J. and Seely Brown, J. (2001) “Your Next IT Strategy”, Harvard Business Review, Vol. 79, No. 9,<br />

October, pp 105–113.<br />

Haki, M. and Forte, M. (2010) “Proposal of a Service Oriented Architecture Governance Model to Serve as a<br />

Practical Framework for Business-IT Alignment”, Proceedings of the 4th Internati<strong>on</strong>al <str<strong>on</strong>g>C<strong>on</strong>ference</str<strong>on</strong>g> <strong>on</strong> New<br />

Trends in Informati<strong>on</strong> Science and Service Science, May, pp 410 – 417.<br />

Hau, T. et al (2008) “Where to start with SOA: Criteria for Selecting SOA Projects”, Proceedings of the 41st<br />

Hawaii Internati<strong>on</strong>al <str<strong>on</strong>g>C<strong>on</strong>ference</str<strong>on</strong>g> <strong>on</strong> System Sciences, USA, January.<br />

Kohli, R. and Grover, V. (2008) “Business Value of IT: An Essay <strong>on</strong> Expanding Research Directi<strong>on</strong>s to Keep up<br />

with the Times”, Journal of the AIS, Vol. 9, No. 1, January, pp 23-37.<br />

K<strong>on</strong>togiannis, K., Lewis G.A. and Smith D.B. (2008) “A Research Agenda for Service-oriented Architecture”,<br />

Proceedings of the 2nd Internati<strong>on</strong>al Workshop <strong>on</strong> Systems Development in SOA Envir<strong>on</strong>ments, May,<br />

Leipzig, Germany.<br />

Luthria, H. and Rabhi, F.A. (2008) “Service Oriented Computing in Practice - An Agenda for Research into the<br />

Factors Influencing the Organizati<strong>on</strong>al Adopti<strong>on</strong> of Service Oriented Architectures”, Journal of Theoretical<br />

and Applied Electr<strong>on</strong>ic Commerce Research, Vol. 4, No. 1, April, pp 39-56.<br />

265


R<strong>on</strong>an O’Sullivan, Tom Butler and Philip O’Reilly<br />

Luthria, H., Rabhi, F.A. and Briers, M. (2007) “Investigating the Potential of Service Oriented Architectures to<br />

Realize Dynamic Capabilities”, Asia-Pacific Service Computing <str<strong>on</strong>g>C<strong>on</strong>ference</str<strong>on</strong>g>, The 2nd IEEE, December, pp<br />

390-397.<br />

Mata, F.J., Fuerst, W. L. and Barney, J. B. (1995) “Informati<strong>on</strong> Technology and Sustained Competitive<br />

Advantage: A Resource-Based Analysis”, MIS Quarterly, Vol. 19, No. 4, December, pp 487-505.<br />

Maurizio, A. et al (2008) “Service Oriented Architecture: Challenges for Business and Academia”, Proceedings of<br />

the 41st Hawaii Internati<strong>on</strong>al <str<strong>on</strong>g>C<strong>on</strong>ference</str<strong>on</strong>g> <strong>on</strong> System Sciences, USA, January.<br />

Melville, N., Kraemer, K. and Gurbaxani, V. (2004) "Review: Informati<strong>on</strong> Technology and Organizati<strong>on</strong>al<br />

Performance: An Integrative Model of IT Business Value", MIS Quarterly, Vol. 28, No. 2, June, pp 283-322.<br />

Merrifield, R., Calhoun J. and Stevens D. (2008) “The Next Revoluti<strong>on</strong> in Productivity”, Harvard Business Review,<br />

June, pp 73-80.<br />

Nevo, S. and Wade, M.R. (2010) “The Formati<strong>on</strong> and Value of IT Enabled Resources: Antecedents and<br />

C<strong>on</strong>sequences of Synergistic Relati<strong>on</strong>ships”, MIS Quarterly, Vol. 23, No. 1, March, pp 163-183.<br />

Papazoglou, M.P. et al (2007) “Service-oriented Computing: State of the Art and Research Challenges”,<br />

Computer 2007, Vol. 40, No. 11, November, pp 38-45.<br />

TechTarget and Forrester Research (2010) “State of SOA 2010”,<br />

http://cdn.ttgtmedia.com/searchSOA/downloads/TTAG-State-of-SOA-2010-execSummary-working-<br />

523%5B1%5D.pdf<br />

Teece D., Pisano, G. and Shuen, A. (1997) “Dynamic Capabilities and Strategic <strong>Management</strong>”, Strategic<br />

<strong>Management</strong> Journal, Vol. 7, No. 18, August, pp 509-533.<br />

Viering, G., Legner, C. and Ahlemann, F. (2009) "The (Lacking) Business Perspective <strong>on</strong> Soa–Critical Themes in<br />

Soa Research", Wirtschaftinformatik Proceedings 2009. Paper 5.<br />

Wernerfelt, B. (1984) "A Resource-based View of the Firm", Strategic <strong>Management</strong> Journal, Vol. 5, No. 2,<br />

April-June, pp 171-180.<br />

Wheeler, B. (2002) "NEBIC: A Dynamic Capabilities Theory for Assessing Net-enablement", Informati<strong>on</strong><br />

Systems Research, Vol. 13, No. 2, June, pp 125-146.<br />

266


The Identificati<strong>on</strong> of Service Oriented Architecture-Specific<br />

Critical Success Factors<br />

Ian Owens and John Cunningham<br />

Cranfield Defence and Security, Shrivenham, UK<br />

i.owens@cranfield.ac.uk<br />

Abstract: This paper reports <strong>on</strong> a research project that sought to determine whether it is possible to identify<br />

Service Oriented Architecture (SOA) specific critical success factors (CSFs). SOA is an approach to designing<br />

interoperable informati<strong>on</strong> systems based <strong>on</strong> a set of design principles and the c<strong>on</strong>cept of loosely coupled<br />

services. In recent years SOA has become the de facto method for designing distributed interoperable<br />

informati<strong>on</strong> systems. Despite the widespread use of SOA design principles, it remains not <strong>on</strong>ly technically difficult<br />

to implement, but also presents a substantial challenge to systems architects and managers. Our hope is that<br />

SOA-specific CSFs will enable project managers involved in SOA implementati<strong>on</strong>s to best allocate resources to<br />

those areas that are critical to the success of SOA-based projects. We c<strong>on</strong>ducted a comprehensive systematic<br />

review of the SOA literature, and identified five SOA-specific CSFs which we believe may be critical for realizing<br />

the benefits of SOA. To externally validate the CSFs identified from our literature review, we surveyed project<br />

managers and implementers in a department in a large defence-related world-wide organizati<strong>on</strong> which is<br />

currently implementing SOA-based systems. The results of our research c<strong>on</strong>firmed the validity of the SOA<br />

specific CSFs identified through our literature review. We recommend that project managers use both our SOAspecific<br />

CSFs and generic project CSFs in combinati<strong>on</strong> to manage SOA projects.<br />

Keywords: SOA, CSFs, project management, enterprise architectures<br />

1. Introducti<strong>on</strong><br />

In the past, large organizati<strong>on</strong>s developed or commissi<strong>on</strong>ed the development of a broad range of<br />

software applicati<strong>on</strong>s from a variety of vendors. The majority of these applicati<strong>on</strong>s were procured for a<br />

specific purpose, often in isolati<strong>on</strong> from other c<strong>on</strong>current developments. Over time, heterogeneous<br />

informati<strong>on</strong> systems (IS) and applicati<strong>on</strong>s increased in both number and complexity the result was an<br />

enormous amount of duplicati<strong>on</strong> and c<strong>on</strong>sequently a great deal of unnecessary and excessive<br />

investment (Tsai 2005). Much of that duplicati<strong>on</strong> of capability was attributable to the fact that many of<br />

these applicati<strong>on</strong>s were incompatible with other IS and thus the number of users of the applicati<strong>on</strong><br />

was restricted to those within that department or using the same hardware or operating system. The<br />

phrase ‘tightly coupled’ was often used to describe applicati<strong>on</strong>s that were highly reliant <strong>on</strong> a particular<br />

system. C<strong>on</strong>versely, the term ‘loosely coupled’ implies no or very little reliance <strong>on</strong> a particular system<br />

(Kaye 2003, Tsai 2005).<br />

Given the cost and commitment of resources required for developing bespoke software, re-use and<br />

systems interoperability have been primary goals of many organizati<strong>on</strong>s, “especially those that rely<br />

heavily <strong>on</strong> computer networks” (Lim and Wen, 2003). Service Oriented Architecture (SOA) is an<br />

architectural pattern that can be used to build distributed IS that allows the use, and most importantly<br />

the re-use, of existing applicati<strong>on</strong>s. SOA not <strong>on</strong>ly leverages <strong>on</strong> existing assets but, by virtue of the<br />

loosely coupled applicati<strong>on</strong>s, SOA facilitates the adding or removing of individual applicati<strong>on</strong>s and<br />

thus makes implementing changes to support changing business requirements simple, quick and<br />

relatively inexpensive.<br />

1.1 Aims of the research<br />

The aims of this research were twofold. Firstly we wanted to determine whether it is possible to<br />

identify SOA specific critical success factors (CSFs). CSFs are an important tool for those planning,<br />

managing and implementing projects. There has been a lot of published work in the field of identifying<br />

CSFs for influencing IS project management outcomes, notably the work of Pinto and Prescot (1988).<br />

Our sec<strong>on</strong>d aim was to build <strong>on</strong> the work of Pinto and Prescott and empirically test SOA-related CSFs<br />

that we derived from theory, <strong>on</strong> a real SOA implementati<strong>on</strong> project.<br />

1.2 Structure of the paper<br />

The paper is structured as follows. We begin by describing SOA and critical success factors. This is<br />

followed by an identificati<strong>on</strong> of SOA-specific CSFs. We then describe our empirical study and discuss<br />

our results. The c<strong>on</strong>cluding secti<strong>on</strong> discusses the relevance of our work and makes some<br />

recommendati<strong>on</strong>s based <strong>on</strong> our findings.<br />

267


2. Service oriented architecture (SOA)<br />

Ian Owens and John Cunningham<br />

SOA is a well established approach to designing informati<strong>on</strong> systems based <strong>on</strong> c<strong>on</strong>cepts such as<br />

reuse and loose coupling of services that promote and enable good IT management. The three main<br />

comp<strong>on</strong>ents of a SOA system are service providers, service c<strong>on</strong>sumers and service<br />

brokers/registries. Service providers are the hosts of services, service c<strong>on</strong>sumers are the users of<br />

services, and service brokers/registries help to facilitate the discovery and advertisement of services,<br />

by utilising a registry (Foster et al. (2002), Papazoglou. and Georgakopoulos (2003), Bucur and<br />

Bardram (2007), Yang and Joy (2010)). Figure 1 illustrates the relati<strong>on</strong>ship between the three<br />

comp<strong>on</strong>ents:<br />

Figure 1: Three comp<strong>on</strong>ents of a SOA-based system.<br />

Van Halteren and Pawar (2006) define SOA as, “essentially a collecti<strong>on</strong> of services that communicate<br />

with each other to achieve a comm<strong>on</strong> goal”. In OASIS (2006) SOA is defined as: “SOA is a paradigm<br />

for organizing and utilizing distributed capabilities that may be under the c<strong>on</strong>trol of different ownership<br />

domains. It provides a uniform means to offer, discover, interact with and use capabilities to produce<br />

desired effects c<strong>on</strong>sistent with measurable prec<strong>on</strong>diti<strong>on</strong>s and expectati<strong>on</strong>s.”<br />

2.1 Critical success factors<br />

The emergence of CSFs as a c<strong>on</strong>cept can be traced back to the work of Daniel (1961) who<br />

introduced the idea of success factors into management literature. In the early 1970s Anth<strong>on</strong>y, et al,<br />

(1972) introduced the idea of tailoring critical success factors, and Rockart, (1979) combined the work<br />

of these authors to describe a study of three organizati<strong>on</strong>s in the same sector. Rockart’s study<br />

c<strong>on</strong>firmed that organizati<strong>on</strong>s in the same industrial sector shared similar CSFs. Rockart decribes<br />

CSFs as:<br />

“Critical success factors thus are, for any business, the limited number of areas in which<br />

results, if they are satisfactory, will ensure successful competitive performance for the<br />

organizati<strong>on</strong>. They are the few key areas where "things must go right" for the business to<br />

flourish.” Rockart, (1979)<br />

Rockart’s definiti<strong>on</strong> is particularly relevant as it has stood the test of time. Even today, Rockart is<br />

routinely quoted in numerous articles related to CSFs for example, Jenster, (1987), Chen, (1999),<br />

Fortune & White, (2006). The 1990’s saw the development of CSFs from strategic decisi<strong>on</strong> making to<br />

the specific domain of Project <strong>Management</strong> (PM). Slevin and Pinto (1986), developed the ‘Project<br />

Implementati<strong>on</strong> Profile’ (PIP) which identified 11 CSFs relevant to a multitude of projects within the<br />

business arena. These CSFs were later tested empirically by Pinto and Prescott (1988); Pinto’s work<br />

was subsequently incorporated into the Project <strong>Management</strong> Institute’s, (1996), ‘Body of Knowledge’<br />

(PMBOK) publicati<strong>on</strong>. Slevin and Pinto’s PIP has been tested empirically by other authors, including<br />

Finch (2003) who evaluated the PIP in an informati<strong>on</strong> systems project.<br />

Our c<strong>on</strong>tenti<strong>on</strong> is that CSFs are an established c<strong>on</strong>cept for achieving successful outcomes to<br />

complex problems and with SOA being a significant undertaking for any organizati<strong>on</strong> it is perhaps<br />

also fair to assume that there will be CSFs associated solely with implementing a SOA. The next<br />

secti<strong>on</strong> describes how we examined existing theory relating to SOA-based IS implementati<strong>on</strong>s to<br />

derive a set of SOA-specific CSFs.<br />

268


3. The discovery of SOA-specific CSFs<br />

Ian Owens and John Cunningham<br />

In order to establish a definitive as possible list of CSFs specific to SOA, We followed the advice of<br />

Webster and Wats<strong>on</strong> (2002) and created a c<strong>on</strong>cept matrix to identify recurring c<strong>on</strong>cepts relating to<br />

SOA implementati<strong>on</strong>s. We c<strong>on</strong>ducted a search using standard databases such as SCOPUS and Web<br />

of Science. In order to decide which of the c<strong>on</strong>cepts are ‘critical’, a mechanism for organizing them<br />

into priority order was required. To achieve this, we subjected the findings from the literature search to<br />

a process of c<strong>on</strong>tent analysis and frequency analysis; and from this applied ‘weightings’ to the results.<br />

Michael and Lewis, (1994) describe c<strong>on</strong>tent analysis as “seeking to draw valid reas<strong>on</strong>ing from text”<br />

based <strong>on</strong> data and c<strong>on</strong>text. Seaman, (1994) commends frequency analysis as a method of “drawing<br />

quantitative data from qualitative data” which affords empirical evidence a degree of statistical<br />

analysis.<br />

This analysis took the form of awarding two scores out of five for each of the candidate CSFs based<br />

<strong>on</strong> the number of occurrences they had and the degree of importance they were given in a number of<br />

articles pertaining to the implementing a SOA. In the case of frequency, the score was based <strong>on</strong><br />

either the frequency with which a particular CSF was discussed, the percentage of text devoted to<br />

that CSF, or a combinati<strong>on</strong> of both. For example, Bieberstein et al, (2008), devotes an entire chapter<br />

to governance and c<strong>on</strong>tinually refers to governance in the other chapters; this merits a score of five<br />

out of five. On the other hand, Roshen et al, (2009) makes no reference whatsoever to governance<br />

and thus merits a score of zero. For the c<strong>on</strong>tent analysis, the mark out of five depends <strong>on</strong> how much<br />

emphasis was given to the task in questi<strong>on</strong> being critical to the successful implementati<strong>on</strong> of SOA.<br />

For example; Lee, et al., (2005), actually state governance as a SOA specific CSF. This therefore<br />

merits a c<strong>on</strong>tent score of five out of five in order to reflect the very str<strong>on</strong>g opini<strong>on</strong> that governance is<br />

indeed a CSF. In the absence of a statement of criticality, the overall score was based <strong>on</strong> the degree<br />

of implied importance the authors of the article appeared to give the task. C<strong>on</strong>tent analysis was<br />

therefore heavily weighted by our own understanding of what was being articulated in the literature.<br />

The findings of the sub-review are shown in Table 1. However, for ease of reference a summary of<br />

the findings is shown in Table 2<br />

Table 1: Critical success factors for implementing a SOA<br />

Critical Success Factors for Implementing SOA<br />

Author<br />

Implementa<br />

ti<strong>on</strong><br />

Methodolog<br />

y<br />

B<strong>on</strong>net, et al.,<br />

(2009)<br />

Brown, (2008)<br />

Erl, (2007)<br />

Rosen, (2008)<br />

Bieberstein, et al.,<br />

(2008)<br />

Roshen, (2009)<br />

Ericks<strong>on</strong> and Siau,<br />

(2008)<br />

Mansukhani,<br />

(2005).<br />

Kavis, (2007)<br />

Lawler, et al.,<br />

(2007)<br />

Figlin, (2008).<br />

Varadan, et al.,<br />

(2008).<br />

Lee, et al., (2010).<br />

Antikainen and<br />

Pekkola, Papazoglou (2009) & van<br />

den Heuvel,<br />

(2007) Gulledge, & Deller,<br />

(2009).<br />

Lim, Wen, (2003)<br />

F 5 2 4 5 5 4 1 4 0 1 5 2 1 0 1 3 2 1<br />

C 5 1 5 4 5 4 3 5 0 1 5 2 1 0 2 4 2 3<br />

Business<br />

Processes<br />

F 5 5 5 3 3 4 3 4 3 4 4 2 3 1 3 5 4 4<br />

Modelling C 5 4 4 3 4 4 3 4 4 4 4 3 3 2 1 5 2 5<br />

Organizatio<br />

F 2 3 2 4 3 0 4 3 5 5 5 4 5 5 4 3 3 5<br />

nal Change<br />

C 4 4 1 3 5 0 5 4 5 5 5 3 5 5 4 3 5 5<br />

Governanc<br />

e<br />

Re-use /<br />

Leverage<br />

F 3 5 5 5 5 0 5 5 5 5 5 4 5 5 4 3 5 5<br />

C 3 5 3 5 5 0 5 5 5 5 5 5 5 5 5 5 5 5<br />

F 2 3 3 3 3 1 2 2 1 0 1 3 1 2 3 4 3 2<br />

C 2 2 2 4 3 2 2 3 1 0 1 4 1 3 3 2 1 2<br />

269<br />

Hochstein, (2005),<br />

Sub Totals<br />

4<br />

6<br />

5<br />

1<br />

6<br />

5<br />

6<br />

4<br />

6<br />

5<br />

7<br />

1<br />

7<br />

8<br />

8<br />

1<br />

3<br />

9<br />

3<br />

8<br />

Totals<br />

9<br />

7<br />

1<br />

2<br />

9<br />

1<br />

3<br />

6<br />

1<br />

5<br />

9<br />

7<br />

7<br />

Rank<br />

4<br />

3<br />

2<br />

1<br />

5


Table 2: Top 5 SOA-Specific CSFs<br />

Ian Owens and John Cunningham<br />

CSF Points Rank<br />

Governance 169 1<br />

Organizati<strong>on</strong>al Change 136 2<br />

Business Process<br />

Modelling/<strong>Management</strong><br />

129 3<br />

Implementati<strong>on</strong> Methodology 97 4<br />

Re-use/Leverage 77 5<br />

4. Company X survey<br />

In order to evaluate our SOA-specific CSFs listed in Table 2 we c<strong>on</strong>ducted an empirical study with an<br />

organizati<strong>on</strong> who are in the process of implemented a large-scale, multi-nati<strong>on</strong>al SOA-based IS<br />

project. For the purposes of this paper the organizati<strong>on</strong> is referred to as company X. Company X is<br />

involved in military-related logistics and they are implementing SOA to rati<strong>on</strong>alise and manage their<br />

heterogeneous legacy systems envir<strong>on</strong>ment. Their chosen approach is to first establish their<br />

informati<strong>on</strong> needs, and then match the need to their current legacy systems. They will then decide<br />

what current systems they want to keep and which are to be discarded. The systems that remain will<br />

be wrapped and the relevant functi<strong>on</strong>ality will then be exposed as services. New services can then be<br />

developed over time to replace those provided by the legacy systems.<br />

We c<strong>on</strong>ducted a survey of the team in company X who are resp<strong>on</strong>sible for managing the<br />

implementati<strong>on</strong> of the SOA approach. A total of 96 questi<strong>on</strong>naires were sent to staff in company X.<br />

This yielded a resp<strong>on</strong>se of 55 completed surveys, or a 57% resp<strong>on</strong>se rate. The results of this survey<br />

are presented and discussed below.<br />

5. Results and discussi<strong>on</strong><br />

Unfortunately there is not enough space in this report to present the results of the entire survey. For<br />

the purposes of this paper we will focus our discussi<strong>on</strong> <strong>on</strong> the results of two questi<strong>on</strong>s that sought to<br />

test the validity of our CSFs and to rank the CSFs in order of priority. Table 3 shows the results from<br />

questi<strong>on</strong> 3 which sought to determine whether the SOA-specific CSFs identified from the literature<br />

were relevant CSFs. Over 75% of resp<strong>on</strong>dents either agreed or str<strong>on</strong>gly agreed that the suggested<br />

tasks were indeed CSFs for SOA implementati<strong>on</strong>, for all variables except Re-use & Leverage which<br />

scored 62.5%.<br />

Table 3: Results of questi<strong>on</strong> 3<br />

How much do you agree that the following activities are Critical Success Factors for the<br />

implementati<strong>on</strong> of a SOA in a large enterprise such as Log NEC?<br />

Str<strong>on</strong>gly Agree Agree Neutral Disagree Str<strong>on</strong>gly Total<br />

SOA Critical<br />

Success Factor<br />

Disagree<br />

Implementati<strong>on</strong><br />

90 100 45 0 0 135<br />

Methodology<br />

(18) (25) (15)<br />

Business<br />

Process 120 80 27 0 0 227<br />

Modelling<br />

(24) (20) (9)<br />

Governance<br />

155 68 27 0 0 250<br />

(31) (17) (9)<br />

Organizati<strong>on</strong>al<br />

95<br />

96 30 0 0 221<br />

Change<br />

(19) (24) (10)<br />

Reuse<br />

55<br />

76 72 0 0 203<br />

/ Leverage<br />

(11) (19) (24)<br />

Table 4 shows the results from Questi<strong>on</strong> 4 which sought to rank the CSFs in order of importance; the<br />

coded scores are shown in bold and the actual number of ‘votes’ for each CSF are shown in brackets.<br />

The 55 resp<strong>on</strong>dents each had five votes which gave a maximum of 275. In some cases, resp<strong>on</strong>dents<br />

gave a particular priority to more than <strong>on</strong>e CSF; therefore, each column has a different total.<br />

The results of the survey compared with the results of our analysis of the literature are shown in table<br />

5.<br />

270


Table 4: Results of questi<strong>on</strong> 4<br />

Ian Owens and John Cunningham<br />

Please rank these proposed SOA Critical Success Factors in order of importance; 1 = most important, 5 =<br />

least important. You may award some, or all, equal priority:<br />

1 2 3 4 5 D<strong>on</strong>’t Kn ow Total<br />

SOA Critical<br />

Success Factor<br />

Implementati<strong>on</strong><br />

50 24 39<br />

14<br />

13<br />

6<br />

Methodology (10) (6) ( 13) ( 7) ( 13)<br />

140<br />

Business Process<br />

45 96 30<br />

10<br />

0 7<br />

Modelling (9) (24)<br />

(10)<br />

( 5) ( 0)<br />

181<br />

Governance 110 44 27<br />

8 1 6<br />

(24) (11) ( 9) ( 4) ( 1)<br />

200<br />

Organizati<strong>on</strong>al<br />

40 8 45<br />

34<br />

4 9<br />

Change (8) (2) (15)<br />

(17)<br />

( 4)<br />

131<br />

Reuse 40 4 24<br />

22<br />

18<br />

9<br />

/ Leverage (8) (1)<br />

(8)<br />

(11)<br />

(18) 108<br />

Table 5 Company X compared with the results of the literature review<br />

CSF Company X Rank Literature Rank<br />

Governance 1 1<br />

Business Process Modelling 2 5<br />

Organizati<strong>on</strong>al Change 3 2<br />

Implementati<strong>on</strong> Methodology 4 4<br />

Re-use/Leverage 5 3<br />

The results of the survey provide some credible empirical evidence to support the validity of the CSFs<br />

we identified from the literature. The difference in the rank of importance of these CSFs is minor and<br />

can be explained by the unique positi<strong>on</strong> of company X. We would expect all SOA implementati<strong>on</strong>s to<br />

differ in terms of maturity of the project, scale of the project, and experience of the project<br />

implementati<strong>on</strong> team. Therefore, we would expect different teams to place different importance <strong>on</strong> the<br />

CSFs, furthermore we would expect rankings of importance to change as the project itself matures.<br />

We also acknowledge that our coding methodology for the literature review was inherently subjective<br />

and this can also explain the differences in the rankings.<br />

6. C<strong>on</strong>clusi<strong>on</strong>s and recommendati<strong>on</strong>s<br />

We set out to determine whether it was possible to identify SOA-related CSFs from the literature, and<br />

to externally validate these CSFs using an empirical study of an implementati<strong>on</strong> of a SOA based<br />

system. In c<strong>on</strong>clusi<strong>on</strong> we can say that we have fulfilled these two objectives. Our systematic study of<br />

the literature identified five CSFs that were specific to SOA implementati<strong>on</strong> projects, as opposed to<br />

general project management CSFs which are applicable to all projects. Our survey of implementati<strong>on</strong><br />

professi<strong>on</strong>als in Company X has provided some external validati<strong>on</strong> of these CSFs. Our research adds<br />

to the rich body of work <strong>on</strong> project management CSFs and we hope that our SOA-specific CSFs will<br />

provide useful guidance for project managers resp<strong>on</strong>sible for future SOA implementati<strong>on</strong> projects. We<br />

would recommend that additi<strong>on</strong>al work is required to replicate and further validate our findings. The<br />

systematic literature survey should be repeated by an independent team of researchers and we would<br />

welcome a repeat of our survey <strong>on</strong> other implementati<strong>on</strong> projects.<br />

References<br />

Anth<strong>on</strong>y, R.N., Dearden, J. and Vancil, R.F., (1972). Key Ec<strong>on</strong>omic Variables, <strong>Management</strong> C<strong>on</strong>trol Systems,<br />

Homewood, Irwin, IL, pp. 138-43.<br />

Antikainen, J. and Pekkola, S. (2009). Factors Influencing the Alignment of SOA Development with Business<br />

Objectives, 17 th <str<strong>on</strong>g>European</str<strong>on</strong>g> <str<strong>on</strong>g>C<strong>on</strong>ference</str<strong>on</strong>g> <strong>on</strong> Informati<strong>on</strong> Systems.<br />

Bieberstein, N., Laird, R. G., J<strong>on</strong>es, K. And Mitra, T., (2008). Executing SOA: A Practical Guide for the Service-<br />

Oriented Architect, IBM Press.<br />

B<strong>on</strong>net P., Detavernier J.M. and Vauquier D., (2009). Sustainable IT Architecture, Wiley, L<strong>on</strong>d<strong>on</strong>.<br />

Brown, P., (2008). Implementing SOA: Total Architecture in Practice, Addis<strong>on</strong>-Wesley. Upper-Saddle River, NJ.<br />

Bucur, D. and Bardram, J. E. (2007), "Resource discovery in activity-based sensor networks", Mob.Netw.Appl.,<br />

vol. 12, no. 2-3, pp. 129-142.<br />

Chen, T. Y., (1999). Critical Success Factors for Various Strategies in the Banking Industry, Internati<strong>on</strong>al Journal<br />

of Bank Marketing, Volume 17, Issue No 2, pp.83-92.<br />

Daniel, D.R. (1961), "<strong>Management</strong> informati<strong>on</strong> crisis", Harvard Business Review, September/October, pp.111-21.<br />

271


Ian Owens and John Cunningham<br />

Erl, T., (2007). SOA: Principles of Service Design. Upper Saddle Creek, NJ, Prentice Hall.<br />

Erics<strong>on</strong>, J and Siau, K., (2008). Critical Success Factors in SOA Implementati<strong>on</strong>, Americas <str<strong>on</strong>g>C<strong>on</strong>ference</str<strong>on</strong>g> <strong>on</strong><br />

Informati<strong>on</strong> Systems, Proceedings, Paper 107.<br />

Figlin, O., (2008). Enterprise SOA Made Easy: Key Success Factors, Available at:<br />

http://www.sap.com/community/webcast/2008_06_10_NetWeaver_CIS/20-<br />

08_06_10_NetWeaver_18_Figlin.pdf. Accessed, 2 July 2011).<br />

Finch, P. (2003). Applying the Slevin-Pinto project management profile to an informati<strong>on</strong> systems project. Project<br />

M anagement Journal, 34(3), 32-9.<br />

Fortune, J. and White, D., (2006). Framing of Project Critical Success Factors by a Systems Model, Internati<strong>on</strong>al<br />

Journal of Project <strong>Management</strong>, Vol. 24, pp. 53-65.<br />

Foster, I., Kesselman,C., Nick,J., Tuecke, S.. (2002). Grid Services for Distributed System Integrati<strong>on</strong>. Computer<br />

35, 6 (June 2002), 37-46.<br />

Gulledge, T. and Deller, G., (2009). Service-Oriented C<strong>on</strong>cepts: Bridging Between Managers and Technologists.<br />

Industrial <strong>Management</strong> & Data Systems, Volume 109, Issue No 1, pp.5–15.<br />

Hochstein, A., Tamm, G. and Brenner, W. (2005). Service-Oriented IT <strong>Management</strong>: Benefit, Cost and Success<br />

Factors. Paper presented at the Thirteenth <str<strong>on</strong>g>European</str<strong>on</strong>g> <str<strong>on</strong>g>C<strong>on</strong>ference</str<strong>on</strong>g> <strong>on</strong> Informati<strong>on</strong> Systems, Regensburg,<br />

Germany.<br />

Jenster, V., (1987). Using Critical Success Factors in Planning. L<strong>on</strong>g Range Planning, Volume 20. Issue No 4,<br />

pp.102-109.<br />

Kavis, M., (2007). SOA Critical Success Factors. Available at: http://it.toolbo-x.com/blogs/madgreek/soa-criticalsuccess-factors-21189,<br />

(Accessed: 30 June 2011).<br />

Kaye, D., (2003). Loosely Coupled: The Missing Pieces of Web Services. RDS Ptes, CA, 2003.<br />

Lawler, J. P., Benedict, V., Howell-Barber, H. and Joseph, A., (2009). Critical Success Factors in the Planning of<br />

a Service-Oriented Architecture (SOA) Strategy for Educators and Managers, Informati<strong>on</strong> Systems<br />

Educati<strong>on</strong>al Journal, Volume 7, Issue No 94.<br />

Lee, J. H., Shim, H. and Kim, K. K., (2010). Critical Success Factors in SOA Implementati<strong>on</strong>: An Exploratory<br />

Study, Informati<strong>on</strong> Systems Journal, Volume 27 Issue No 2, p123-145<br />

Lim, B., Wen, J., 2003. Web Services: An Analysis of the Technology, its Benefits, and Implementati<strong>on</strong><br />

Difficulties. Journal of Informati<strong>on</strong> Systems <strong>Management</strong>, Spring, pp.49–57.<br />

Papazoglou, M. P. and Georgakopoulos, D. (2003), "Service-oriented computing", Communicati<strong>on</strong>s of the ACM,<br />

vol. 46, no. 10, pp. 25-28.<br />

Mansukhani, M. (2005). Service Oriented Architecture - Whitepaper, Hewlett-Packard Dev. Comp., Tech. Rep.<br />

Available at http://h20219.www2.hp.co-m/services/downloads/soa_wp_062005.pdf. (Accessed 23 June<br />

2011).<br />

Michael, S. and Lewis, B. 1994. “Research practice: Internati<strong>on</strong>al handbooks of quantitative applicati<strong>on</strong> in social<br />

sciences”, Sage, 1994<br />

OASIS. Reference model for service oriented architecture 1.0 OASIS standard, 12 october 2006. C. Matthew<br />

MacKenzie, Ken Laskey, Francis McCabe, Peter F. Brown, and Rebekah Metz (eds.), http://docs.oasisopen.org/soa-rm/v1.0/soa-rm.pdf<br />

Papazoglou, M. P. and Van Den Heuvel, W. (2006). Service oriented design and development methodology. Int.<br />

J. Web Eng. Technol. 2, 4 (July 2006), 412-442.<br />

Pinto, J.K., Prescott, J.E., (1988). Variati<strong>on</strong>s in Critical Success Factors over the Stages in the Project Life Cycle,<br />

Journal of <strong>Management</strong>, Volume14 , Issue No1, pp.5–18.<br />

Pinto, J.K., & Slevin, D.P., (1987). Critical Factors in Successful Project Implementati<strong>on</strong>, IEEE, Transacti<strong>on</strong>s <strong>on</strong><br />

Engineering <strong>Management</strong>, Volume 34, pp.22-27.<br />

Rockart, J.F., (1979). Chief Executives Define Their Own Data Needs, Harvard Business Review, Vol. 57, Issue<br />

No. 2, pp.238-41.<br />

Rosen, M., Lublinski, B., Smith, K. and Balcer, M., (2008). Applied SOA: Service Oriented Architecture and<br />

Design Strategies, Wiley, IN, USA.<br />

Roshen, W., (2009). SOA-BASED Enterprise Integrati<strong>on</strong>, McGraw-Hill, USA<br />

Seaman, C.B. (1994), "Using the OPT Improvement Approach in the SQL/DS Development Envir<strong>on</strong>ment". In<br />

Proceedings of CASCON'94, (CD-ROM versi<strong>on</strong>), Tor<strong>on</strong>to, Canada, October.<br />

Tsai, W. T., (2005). Service-Oriented System Engineering: A New Paradigm. IEEE Internati<strong>on</strong>al Workshop <strong>on</strong><br />

Service-Oriented System Engineering (SOSE 05), pp.3–6, October 2005.<br />

van Halteren, A.T. and Pawar, P. (2006) Mobile Service Platform: A Middleware for Nomadic Mobile Service<br />

Provisi<strong>on</strong>ing. In: IEEE Internati<strong>on</strong>al <str<strong>on</strong>g>C<strong>on</strong>ference</str<strong>on</strong>g> <strong>on</strong> Wireless and Mobile Computing, Networking and<br />

Communicati<strong>on</strong>s, 2006 (WiMob'2006)., 19-21 Jun 2006, M<strong>on</strong>treal, Canada. pp. 292-299. IEEE Computer<br />

Society. ISBN 1-4244-0494-0<br />

Varadan, R., Channabasavaiah, S., Simps<strong>on</strong>, K., Holley, A. and Allam, A., (2008). Increasing Business Flexibility<br />

and SOA Adopti<strong>on</strong> through Effective SOA Governance,’ IBM Systems Journal, Volume 47, Issue No. 3,<br />

pp.473–488.<br />

Webster, J. and Wats<strong>on</strong>, T. R., (2002). Analyzing the Past to Prepare for the Future: Writing a Literary Review,<br />

MIS Quarterly, Volume 26, Issue No 2.<br />

Yang, S. and Joy, M. S. (2010), "Service Advertisement and Discovery", in Griffiths, N. E. and Chao, K. (eds.)<br />

Springer, pp. 21-46.<br />

272


Treasure Hunting in the 21 st century: A Decade of<br />

Geocaching in Portugal<br />

Teresa Santos, Ricardo Mendes, António Rodrigues and Sérgio Freire<br />

e-GEO, Faculdade de Ciências Sociais e Humanas, FCSH, Universidade Nova<br />

de Lisboa, Lisboa, Portugal<br />

teresasantos@fcsh.unl.pt<br />

rnmendes@fcsh.unl.pt<br />

amrodrigues@fcsh.unl.pt<br />

sfreire@fcsh.unl.pt<br />

Abstract: The present study looks at geocaching, a popular locati<strong>on</strong>-based mobile game, where the goal is to<br />

use a Global Navigati<strong>on</strong> Satellite System (GNSS), usually the Global Positi<strong>on</strong>ing System (GPS) to hide and seek<br />

c<strong>on</strong>tainers placed anywhere in the field. People who engage in this activity, the geocachers, c<strong>on</strong>stitute a<br />

geographically distributed community that makes use of mobile and Web 2.0 technologies to coordinate and<br />

document their activities. C<strong>on</strong>sequently, this treasure-hunting game, besides being a ludic activity, associated<br />

with a str<strong>on</strong>g social networking element, also promotes new ways of exploring, interacting and communicating<br />

experiences and percepti<strong>on</strong>s of the geographical envir<strong>on</strong>ment where the activity occurs. The majority of existing<br />

literature analyzes geocaching from a social point of view, and little reference is made to the geographical c<strong>on</strong>text<br />

of this activity. The aim of this study is to fill that gap and thus characterize the phenomen<strong>on</strong> in terms of its<br />

temporal and spatial distributi<strong>on</strong>. Observati<strong>on</strong> instruments are proposed based <strong>on</strong> motorizati<strong>on</strong> indexes built from<br />

available data attributes. Such attributes reveal behaviors and patterns of geocachers (individuals) and<br />

geocaches (objects). The methodology is based <strong>on</strong> spatial data analysis; this can play an important role in<br />

exploring social phenomena that have a str<strong>on</strong>g geographic comp<strong>on</strong>ent. Through the analysis of the freely<br />

available dataset that is voluntarily maintained by people engaged in the geocaching activity, a new dimensi<strong>on</strong> is<br />

explored: the spatial dimensi<strong>on</strong>. When, where and why this activity occurs was used as the framework for the<br />

analysis in this paper. The final output is an overall picture of the geocaching activity in mainland Portugal in this<br />

decade. In a later stage, envir<strong>on</strong>mental characteristics are used as possible explanati<strong>on</strong>s for observed patterns. It<br />

is shown, using spatial model specificati<strong>on</strong>s, that a small number of regressors are able to highlight important<br />

characteristics in the data. A final discussi<strong>on</strong> underlines the potential of geocaching to encourage social<br />

interacti<strong>on</strong>, and promote cultural and natural heritage; in short, it has some paramount attributes of an<br />

ec<strong>on</strong>omically sound and sustainable sector.<br />

Keywords: geocaches, geocaching, GPS, Web 2.0, spatial analyses<br />

1. Introducti<strong>on</strong><br />

The Web 2.0 has brought a huge change in human interacti<strong>on</strong>s in the first decade of the 21 st century.<br />

One of the recent social phenomena of Web 2.0 is geocaching. This 10 year-old activity brings<br />

together a treasure hunting game that: 1) looks like a true sport/open air activity, 2) involves hi-tech<br />

gadgets (from handled GPS units to fancy smartph<strong>on</strong>es or tablets), and 3) includes the Internet!<br />

Altogether, geocaching is a true grownups' playground, where individuals, groups of friends or entire<br />

families can interact. The goal of geocaching is to find a hidden c<strong>on</strong>tainer, named geocache, in a<br />

public place and then sign the logbook to record the visit. The caches’ informati<strong>on</strong> is published in the<br />

official geocaching website (www.geocaching.com), and it includes a GPS positi<strong>on</strong>, al<strong>on</strong>g with some<br />

clues and most often informati<strong>on</strong> regarding the place where the cache is hidden. Any<strong>on</strong>e can register<br />

for free, and the <strong>on</strong>ly requirements are a GNSS unit or a GNSS-enabled device. Usually, but not<br />

always, the log is later registered in the official website and experiences are shared with other<br />

geocachers.<br />

Created in 2000, geocaching reached Portugal in February 2001. Ten years and 18,500 geocaches<br />

later, almost 2,000,000 logs were d<strong>on</strong>e, bringing together a community of over 17,000 registered<br />

individuals just in Portugal.<br />

Geocaching can be seen as a touristic activity. Many caches are placed in heritage sites, and have<br />

detailed descripti<strong>on</strong> and photos of the m<strong>on</strong>ument/site in their webpage. Furthermore, numerous logs<br />

are posted by foreigners. The most visited cache in Portugal has been found 1577 times and is<br />

located in Centro Cultural de Belém, followed by a cache located in Parque das Nações (both in<br />

Lisb<strong>on</strong>), with 1492 findings. The two places are well known touristic spots. In this way, geocaching is<br />

a volunteer activity that can also be used as an attracti<strong>on</strong> factor for travelers to visit certain<br />

destinati<strong>on</strong>s, combining holiday time with the enjoyment of local history, culture and landscape.<br />

273


Teresa Santos et al.<br />

Several touristic projects that include geocache trails were already developed in the United States of<br />

America and Canada, am<strong>on</strong>g other countries.<br />

There are several types of caches (Traditi<strong>on</strong>al, Multi, Mystery, Event, Letterbox, Virtual, Webcam,<br />

Wherigo TM and Earthcache), of different sizes (from micro, or nano-caches to large, virtual or<br />

unknown). Normally each cache c<strong>on</strong>tains a logbook or logsheet, and geocachers are invited to<br />

exchange pers<strong>on</strong>al objects or symbolic gifts that travel from cache to cache. Some of these gifts,<br />

called travel bugs, can even be traceable. The event and mega-event type, are a particular social<br />

phenomen<strong>on</strong>. These caches require a large group of people (up to 500 geocachers or more) to be<br />

present in a specific day for the event; afterwards, the cache is archived. Mystery or virtual caches<br />

may involve puzzles or brainteasers. Other caches, when hidden relatively close or in <strong>on</strong>e particular<br />

trek or trail, are named power trail. Whereigo TM are particular caches that use a toolset for creating<br />

and playing GPS-enabled adventures. These toolsets are already included in many GPS units.<br />

Earthcaches are dedicated to earth sciences or special geological features. Most geocaches are<br />

totally free for the owner to hide in any public place; others, like Earthcaches, should follow specific<br />

rules in order to be officially published.<br />

Although geocaching promotes the explorati<strong>on</strong> and c<strong>on</strong>tact with nature and outdoor activity, it can<br />

also result in high pressure <strong>on</strong> more fragile envir<strong>on</strong>ments. In fact, geocaching is a volunteer activity<br />

that is not yet subject to any type of c<strong>on</strong>trol, and its implicati<strong>on</strong>s and effects <strong>on</strong>, for example, protected<br />

areas recommend a deeper discussi<strong>on</strong> about such outdoor activities. In Portugal, several such<br />

situati<strong>on</strong>s have been detected during this analysis: 8 caches (with over 500 logs – 1 per week) have<br />

been hidden in the total protecti<strong>on</strong> area of the Arrábida Natural Park, 50 km south of Lisb<strong>on</strong>. Other<br />

caches hidden in the vicinity of nesting col<strong>on</strong>ies of endangered species were also identified and<br />

reported to local authorities. Newsome et al (2011) present a study <strong>on</strong> the envir<strong>on</strong>mental impacts of<br />

adventure racing in Australia in protected areas. These include soil erosi<strong>on</strong>, loss of vegetati<strong>on</strong> or<br />

alterati<strong>on</strong> of feeding patterns of local fauna, am<strong>on</strong>g others. Also, human waste can play a negative roll<br />

<strong>on</strong> these habitats. M<strong>on</strong>z et al (2010) look at outdoor recreati<strong>on</strong> as an agent of ecological change with<br />

the potential to affect soil, vegetati<strong>on</strong>, wildlife, and water quality. The authors c<strong>on</strong>clude that the current<br />

research <strong>on</strong> envir<strong>on</strong>mental recreati<strong>on</strong> needs to be extended and include, besides local impact<br />

indicators, aspects of spatial and temporal analysis that will allow improving and bring the study to the<br />

landscape level. As a resp<strong>on</strong>se to these critics, Geocaching.com attempts to introduce some ethics<br />

and rules <strong>on</strong> the geocaching activity by profiling the participants as virtual defenders of the<br />

envir<strong>on</strong>ment, and providing guidelines <strong>on</strong> how to navigate in nature without harming the envir<strong>on</strong>ment<br />

(Gram-Hansen, 2009).<br />

In the literature, Geocaching has been studied as a social phenomen<strong>on</strong> (Gram-Hansen, 2009), that<br />

can be seen as an informal learning tool (Clough, 2010), or as a deviant behavior (Hawley, 2010),<br />

al<strong>on</strong>g with its practice and motivati<strong>on</strong>s (O’Hara, 2008). However, besides being a social activity,<br />

geocaching is also a locati<strong>on</strong>-based experience. The study of the territory and its influence in this<br />

outdoor activity requires therefore the applicati<strong>on</strong> of spatial analysis' methodologies.<br />

In this study, the temporal and spatial disseminati<strong>on</strong> of geocaches in Portugal is analyzed. Key<br />

questi<strong>on</strong>s emerged: “Are caches distributed the same way as natural, anthropic and social<br />

attributes?”, “As an outdoor activity, are there relatively more caches hidden in n<strong>on</strong>-urban areas?”,<br />

“Which are the regi<strong>on</strong>s with more/less geocaches?”, “What is the visiting frequency and intensity per<br />

regi<strong>on</strong>?”, “Which land-uses relate to the locati<strong>on</strong> where most popular caches are placed?”, “Are the<br />

caches’ difficulty levels related with the landscape?”. In sum, all these questi<strong>on</strong>s help to examine what<br />

are the spatial characteristics of the geocaches that make them attractable.<br />

Spatial data analysis can play an important role in studying social phenomena that have a str<strong>on</strong>g<br />

geographic comp<strong>on</strong>ent like geocaching. Through the analysis of the freely-available dataset, webforums<br />

and web pages that are voluntarily maintained by geocachers, a new dimensi<strong>on</strong> is explored:<br />

the spatial dimensi<strong>on</strong>, allowing new insights regarding the relati<strong>on</strong>ships between people and the<br />

surrounding urban and rural envir<strong>on</strong>ment.<br />

2. Study area and data sets<br />

The area selected to study the geocaching activity is mainland Portugal (Figure 1). The country is<br />

divided in several administrative levels. According to the Nomenclature of Territorial Units for<br />

Statistics (NUTS), mainland Portugal comprises 28 NUTS III regi<strong>on</strong>s. Portuguese landscape and<br />

274


Teresa Santos et al.<br />

climate invite to outdoor activities, such as geocaching. Portugal has a very diverse landscape,<br />

varying from mountains in the center to flat plains in the south, and from deep green valleys with<br />

vineyards in the north, to beaches in the coast. Populati<strong>on</strong> density is higher near the coastline,<br />

especially in the metropolitan areas of Oporto and Lisb<strong>on</strong>. Landscape becomes more rural as we<br />

move to the interior. The forest area covers about 38% of the territory, with agriculture land and<br />

farming being the sec<strong>on</strong>d most significant land use. Portugal has a Mediterranean climate and is <strong>on</strong>e<br />

of the <str<strong>on</strong>g>European</str<strong>on</strong>g> countries with the highest levels of annual solar radiati<strong>on</strong>, a factor which favors<br />

outdoor activities.<br />

Data for this study was collected from www.geopt.org (<strong>on</strong>e of the Portuguese geocaching forums) <strong>on</strong><br />

February 2012. The geocaches’ subset includes 18026 caches, hidden between February, 2 nd , 2001<br />

and February, 8 th , 2012. In order to make the analyses more feasible, <strong>on</strong>ly caches located <strong>on</strong> the<br />

mainland were c<strong>on</strong>sidered. For the temporal analysis, the caches with no visiting activity or misplaced<br />

caches (with coordinates outside of the study area) where removed from the data set. The total<br />

number of caches equals 17291. For the geographical analysis, archived caches (caches disabled by<br />

the owner), were also deleted from the database. The final database for geographic analysis c<strong>on</strong>sists<br />

in 13553 operati<strong>on</strong>al caches. This set allows evaluating geocaching for a specific time-stamp. The<br />

attributes included in the data set are cache name, cache owner, locati<strong>on</strong>, difficulty, terrain, cache<br />

type, status, size, hidden date, average size of logs, total number of logs, number of findings, nofindings,<br />

notes, photos, traceable items and votes.<br />

In order to analyze the relati<strong>on</strong> between landscape and geocaching activity, two maps were used. The<br />

Land Use / Land Cover Map of C<strong>on</strong>tinental Portugal (Cartografia de Ocupação do Solo de Portugal<br />

C<strong>on</strong>tinental, IGP 2011) is available for the whole country for 2007, with a minimum mapping unit of 1<br />

ha, and 2 levels of thematic detail. The Natura 2000 network is composed of areas of community<br />

importance for the c<strong>on</strong>servati<strong>on</strong> of habitats and species, in which human activities must be compatible<br />

with the preservati<strong>on</strong> of those natural values (EC, 2012). For the study of populati<strong>on</strong> distributi<strong>on</strong>, the<br />

census tracts for 2011 (preliminary results) were used (INE, 2011).<br />

Figure 1: Study area and data set used for studying the geocaching phenomena at NUTSIII level<br />

3. Methodology<br />

Geocaching is a social networking locati<strong>on</strong>-based game. In the present study, a Geographic<br />

Informati<strong>on</strong> System (GIS) was used to model and analyze the spatial distributi<strong>on</strong> of this activity in<br />

Portugal. In order to take into account the neighborhood effect of geographical data, Exploratory<br />

Spatial Data Analysis (ESDA) and spatial statistics are used (Lloyd, 2011; Olaya, 2012). The former<br />

are particularly important in the present c<strong>on</strong>text, since cross spatial variati<strong>on</strong> is studied in order to<br />

evaluate the relati<strong>on</strong> between locati<strong>on</strong> of the caches and the social and natural c<strong>on</strong>text of the areas<br />

where they are placed. Hence, geographical datasets are aggregated for particular regi<strong>on</strong>s, which<br />

raise comm<strong>on</strong> analytical questi<strong>on</strong>s related to lattice data (Rogers<strong>on</strong>, 2001).<br />

275


Teresa Santos et al.<br />

Geographical phenomena are generally point-based. This means that associated to most acti<strong>on</strong>s,<br />

there is a pair of geographical coordinates which pin-point their locati<strong>on</strong> relative to a particular model<br />

of the Earth. When such data is aggregated and summarized for specific areas, there is a loss of<br />

informati<strong>on</strong>. Spatial stati<strong>on</strong>arity must be assumed since intra-regi<strong>on</strong>al variati<strong>on</strong>s are not known (Lloyd,<br />

2011). Resulting mis-interpretati<strong>on</strong>s are generally related to ecological fallacy (Freedman, 2001). The<br />

other comm<strong>on</strong> issue which should be taken into account when dealing with lattice datasets is that<br />

aggregates are highly dependent <strong>on</strong> the shape of each spatial unit; this is what is known as the<br />

Modifiable Areal Unit Problem (MAUP) In most cases, this may not be c<strong>on</strong>trolled, but should always<br />

be acknowledged (Rogers<strong>on</strong>, 2001).<br />

Assuming spatial phenomena occurs in a c<strong>on</strong>tinuous - although irregular – surface, it is a too str<strong>on</strong>g<br />

assumpti<strong>on</strong> to analyze observati<strong>on</strong>s as independent (Anselin, 1988; Finglet<strong>on</strong>, 1999). Exploratory<br />

Spatial Data Analysis (ESDA) has the merit of making space endogenous. Neighborhood relati<strong>on</strong>s<br />

are quantified and serve as weights in the c<strong>on</strong>structi<strong>on</strong> of spatially lagged variable transformati<strong>on</strong>s. As<br />

an illustrati<strong>on</strong>, for a given variable X, its i th observati<strong>on</strong> ( ) is given by the weighted average of X<br />

observed in the i's k set of neighbors. Formally:<br />

where represent the neighborhood between spatial units i and j.<br />

Spatial Lag transformati<strong>on</strong>s have various uses, which range from making endogenous spatial effects<br />

in a regressi<strong>on</strong> – allowing estimated coefficients to be BLUE (Best Linear Unbiased Estimators)<br />

(Anselin, 1988), to the smoothing of series, facilitating the identificati<strong>on</strong> of geographical clusters. Their<br />

importance increases with the level of global or partial spatial autocorrelati<strong>on</strong> (Getis, 2007). Spatial<br />

autocorrelati<strong>on</strong> may be generally understood as the existing co-variati<strong>on</strong> between observati<strong>on</strong>s al<strong>on</strong>g<br />

any given spatial surface. This is in general terms the result of spatial dependence and heterogeneity.<br />

In the present article, spatial autocorrelati<strong>on</strong> is taken into account in two stages: first, spatial lags for<br />

variables c<strong>on</strong>sidered of significant importance are used to visually analyze patterns at the municipality<br />

level. Sec<strong>on</strong>d, when testing the functi<strong>on</strong>al relati<strong>on</strong>ships which may explain the distributi<strong>on</strong> of the geocaching<br />

phenomen<strong>on</strong>, spatial dependence is made endogenous. In this latter case, spatial<br />

autoregressive and spatial error forms (SAR and SEM) are estimated using maximum likelihood. In<br />

the SAR specificati<strong>on</strong>s, the spatial lag of the dependent variable is used as a regressor; in the SEM<br />

specificati<strong>on</strong>s, spatial dependence is assumed to be captured in the error term. Formally:<br />

SAR: ,<br />

SEM: ,<br />

where is the dependent variable, a matrix of regressors, its regressor, a spatial weights matrix<br />

and the spatial autocorrelati<strong>on</strong> coefficients, and white noise. A set of appropriate tests will be<br />

c<strong>on</strong>ducted in order to infer <strong>on</strong> robustness (R 2 , Log-likelihood, Schwarz and Akaike Informati<strong>on</strong><br />

Criteria) and test the existence of heteroscedasticity (Breusch-Pagan test). Finally, a Likelihood-Ratio<br />

statistic will infer <strong>on</strong> spatial dependence in the series.<br />

In order to explain the distributi<strong>on</strong> of caches over Mainland Portugal, the gross number will not be<br />

used, since this is obviously related to two mass measures associated with municipalities: physical<br />

area and resident populati<strong>on</strong>. Hence, two density metrics are used: number of caches per resident<br />

(C[pop]) and per populati<strong>on</strong> density (c[dpop]). As regressors, two sets of variables are used: <strong>on</strong>e<br />

related to geographical positi<strong>on</strong> in relati<strong>on</strong> to centripetal nodes, and another related to land-use.<br />

In terms of geographical positi<strong>on</strong>, two measures of peripherality are used; <strong>on</strong>e quantifies positi<strong>on</strong> and<br />

mass in relati<strong>on</strong> to the coast (p[coast]) and the other in relati<strong>on</strong> to the two main Portuguese urban<br />

nodes, Lisb<strong>on</strong> and Oporto (p[lp]). The metrics used are taken from Rodrigues (2001). The reas<strong>on</strong>ing<br />

for the use of these two measures is the role of the Atlantic coast as a decisive pooling factor which<br />

helps to explain the existing coastal/interior divide (Ferrão, 2002); also, the Lisb<strong>on</strong> and Oporto<br />

276<br />

,


Teresa Santos et al.<br />

metropolitan areas c<strong>on</strong>centrate most of the country's ec<strong>on</strong>omic activity which make these two nodes<br />

the most important pooling centers of the country.<br />

The sec<strong>on</strong>d set of regressors tries to explain the distributi<strong>on</strong> of caches as being partially dictated by<br />

land-use. The main hypothesis is that users may show preference towards urban areas (which would<br />

be partially explained by their easier accessibility) or alternatively, towards less densely populated<br />

parts of the study-area. With this in mind, the Land Use / Land Cover Map of C<strong>on</strong>tinental Portugal<br />

was used. For each municipality, the proporti<strong>on</strong> of urban area in respect to total area was used as a<br />

regressor (COS[urb]). The same method was used to calculate the proporti<strong>on</strong> of green area<br />

(COS[green]).The exact land-use classes were chosen based <strong>on</strong> the preliminary exploratory analysis<br />

presented below. Finally, Natura 2000 network was used as a final measure of landscape with natural<br />

interest. Again, this was weighted by total area for each municipality (NAT).<br />

4. Results<br />

4.1 When: Temporal analysis of geocaching activity in Portugal<br />

Although geocaching in Portugal has started in 2001, it took 5 years for the activity to take off. After<br />

2006 it has truly exploded, reaching almost 18,000 caches spread all over the country (Figure 2). The<br />

mean growth rate from 2001 to 2011 (2012 was not used in this analysis) is 213% per year. This rate<br />

includes all status of caches: active, “needs maintenance” (caches that are momentarily not available)<br />

and archived.<br />

Figure 2: Yearly evoluti<strong>on</strong> of geocaches hidden in Portugal<br />

4.2 Where: Spatial distributi<strong>on</strong> of geocaches in Portugal<br />

The spatial characterizati<strong>on</strong> of caches in the study area was performed by overlaying the caches<br />

locati<strong>on</strong> with administrative and thematic maps and summarizing their totals.<br />

The spatial distributi<strong>on</strong> of caches in C<strong>on</strong>tinental Portugal by NUTS III shows that there is a str<strong>on</strong>g<br />

presence al<strong>on</strong>g the coast. The Great Lisb<strong>on</strong> area has the highest percentage of the total caches<br />

(14%) followed by the Dão-Lafões area, with 6,7% (Figure 3). The area with the lowest amount of<br />

caches is Serra da Estrela, the highest mountain range in c<strong>on</strong>tinental Portugal, with nearly 1%.<br />

The spatial distributi<strong>on</strong> of geocaches shows some clear patterns. Figure 4 shows: (1) the distributi<strong>on</strong><br />

of the number of caches weighted by resident populati<strong>on</strong>, (2) by total area and (3) by populati<strong>on</strong><br />

density. The original point-dataset was aggregated at the municipalities’ level in order to analyze the<br />

cross-variati<strong>on</strong> taking into account the three mass indicators just menti<strong>on</strong>ed. The resulting densities<br />

were spatially weighted using a row-standardized binary c<strong>on</strong>tiguity spatial weights matrix. The result<br />

is a set of spatial lagged variables.<br />

277


Teresa Santos et al.<br />

When caches are weighted by resident populati<strong>on</strong>, what is observed is a str<strong>on</strong>g corridor running from<br />

the coastal area just north of Lisb<strong>on</strong>, following the north-east directi<strong>on</strong> (northern mountain system<br />

M<strong>on</strong>tejunto-Estrela). When weighted by area, c<strong>on</strong>centrati<strong>on</strong> al<strong>on</strong>g the coast is clear, justified by the<br />

increasing surface area of municipalities as we move inland.<br />

Figure 3: Distributi<strong>on</strong> of Geocaches (NUTSIII and municipalities)<br />

Figure 4: Density of geocaches<br />

278


Teresa Santos et al.<br />

Regarding land cover, caches are placed preferably in natural or semi-natural areas, covered with<br />

scrubs and/or herbaceous vegetati<strong>on</strong> (22,86%), followed by urban areas (19,50%) and forests<br />

(18,85%) (Table 1). This c<strong>on</strong>firms that geocaching is mostly an outdoor activity c<strong>on</strong>ducted outside of<br />

urban centers. Although most people live in artificial areas, most of the caches are placed in natural<br />

envir<strong>on</strong>ments. Based <strong>on</strong> this analysis, two indicators were c<strong>on</strong>structed and used as regressors:<br />

classes 1.1 (“Urban fabric”), <strong>on</strong> <strong>on</strong>e hand, 3.1 (“Forests”) and 3.2 (“Scrub and/or herbaceous<br />

vegetati<strong>on</strong> associati<strong>on</strong>s”) <strong>on</strong> the other, are used and their area per municipality calculated<br />

representing the dichotomy between urban and natural landscapes (represented by COS[urb] and<br />

COS[green]).<br />

Table 1: Distributi<strong>on</strong> of geocaches by land use / land cover class<br />

Geocaches<br />

Land Use / Land Cover Map of 2007 Number Percentage<br />

1.1 Urban fabric 2643 19,50<br />

1.2 Industrial, commercial and transport units 710 5,24<br />

1.3 Mine, dump and c<strong>on</strong>structi<strong>on</strong> sites 132 0,97<br />

1.4 Green urban areas, sports and leisure facilities 228 1,68<br />

2.1 Arable land 1210 8,93<br />

2.2 Permanent crops 606 4,47<br />

2.3 Pastures 142 1,05<br />

2.4 Heterogeneous agricultural areas 1157 8,54<br />

3.1 Forests 2555 18,85<br />

3.2 Scrub and/or herbaceous vegetati<strong>on</strong> associati<strong>on</strong>s 3098 22,86<br />

3.3 Open spaces with little or no vegetati<strong>on</strong> 407 3,00<br />

4.1 Inland wetlands 17 0,13<br />

4.2 Maritime wetlands 56 0,41<br />

5.1 Inland waters 291 2,15<br />

5.2 Marine waters 301 2,22<br />

4.3 Why: Envir<strong>on</strong>mental c<strong>on</strong>diti<strong>on</strong>ing<br />

As described above, besides exploring the distributi<strong>on</strong> of the phenomen<strong>on</strong>, the goal of the study was<br />

to explain the observed patterns. Tables 2 and 3 show the ec<strong>on</strong>ometric results from the spatial lag<br />

(SAR) and the spatial error specificati<strong>on</strong>s (SEM). The variables used were first standardized,<br />

subtracting the mean and dividing by two times the standard deviati<strong>on</strong>, as proposed by Andrew<br />

Gelman (2007). When the number of caches weighted by resident populati<strong>on</strong> is used as the<br />

dependent variable (Model 1), peripherality in relati<strong>on</strong> to Lisb<strong>on</strong> and Oporto (p[lp]) is significant with a<br />

negative sign in the estimated coefficient. COS[urb] is also significant with an opposite sign.<br />

Table 2: Estimati<strong>on</strong> results (SAR specificati<strong>on</strong>)<br />

This shows some tendency of caches to be distributed near urbanized areas. On the other hand,<br />

natural heritage is also positively rated (positive significant coefficient associated with NAT).<br />

When the total number of caches is weighted by populati<strong>on</strong> density (Model 2), results are similar, yet<br />

with some notable differences. First, robustness increases; this is shown by a drastic increase in the<br />

R-squared statistic, smaller absolute value of the Log likelihood functi<strong>on</strong> and Akaike Informati<strong>on</strong><br />

Criteria and Schwarz Criterium. The Likelihood Ratio test also corroborates these results. Sec<strong>on</strong>d,<br />

results of the Breusch-Pagan test indicate that heteroscedasticity is no l<strong>on</strong>ger an issue. In relati<strong>on</strong> to<br />

the estimated coefficients, COS[green] and NAT are both significant with a positive sign, peripherality<br />

279


Teresa Santos et al.<br />

is no l<strong>on</strong>ger relevant whilst COS[urb] is <strong>on</strong>ly significant at the 90% level. In order to get rid of any<br />

multicollinearity, <strong>on</strong>e peripherality measure together with COS[green] were dropped and the model<br />

estimated <strong>on</strong>e last time.<br />

The results are clear and quite attractive: locati<strong>on</strong> of the caches is str<strong>on</strong>ger near the coast. Natural<br />

heritage (NAT) is particularly important with by far the highest coefficient. Yet, although caches tend<br />

to be located within municipalities with attractive landscapes, they generally tend to be in or near<br />

urban areas. Finally, Model 3 excludes the measure of peripherality with the least significance (p[lp]),<br />

as both are highly correlated. For the same reas<strong>on</strong>, COS[green] is also not included. The estimati<strong>on</strong><br />

results c<strong>on</strong>firm previous inferences. Although the percentage of variance explained is smaller (an<br />

expected result given the smaller number of regressors), the unexplained variance is captured by the<br />

autocorrelati<strong>on</strong> coefficient. Results from the SEM specificati<strong>on</strong> (table 3) c<strong>on</strong>firm the SAR estimati<strong>on</strong><br />

results.<br />

Table 3: Estimati<strong>on</strong> results (SEM specificati<strong>on</strong>)<br />

5. C<strong>on</strong>clusi<strong>on</strong>s<br />

This study intended to analyze the nati<strong>on</strong>al scenario of the geocaching activity in Portugal, in terms of<br />

temporal and spatial distributi<strong>on</strong>. The results allowed inferring the evoluti<strong>on</strong> of geocaching in mainland<br />

Portugal; as well as the places where the activity is most prevalent and provided clues to what are the<br />

caches’ characteristics that make them more popular than others.<br />

Due to the lack of literature, this research proposes a methodology and some indexes for m<strong>on</strong>itoring<br />

the geocaching phenomen<strong>on</strong>, allowing for comparis<strong>on</strong> with other countries and other time periods.<br />

Results for the Portuguese dataset show that geo-cachers prefer locati<strong>on</strong>s with significant natural<br />

heritage sites although, <strong>on</strong> average, they tend not to travel far for their treasure-hunting activities.<br />

Cross-variati<strong>on</strong> with other datasets should provide important and highly innovative insights about the<br />

traveling behavior of individuals during their “adventures into the wild”.<br />

Future studies should include other factors that will help to fully understand the phenomena. Local<br />

and regi<strong>on</strong>al trends are surely affected by other aspects like individual motivati<strong>on</strong>s, expectati<strong>on</strong>s and<br />

percepti<strong>on</strong>s; social networking or physical aspects of places where caches are hidden (landscape,<br />

scenic views, cultural heritage, natural phenomena, and so <strong>on</strong>). Other variables like the presence or<br />

absence of structured supply of outdoor activities such as hiking and mountain biking trails, climbing<br />

or speleology, bird watching and other soft nature activities may also explain what seems to be the<br />

huge success of geocaching in Portugal. Naturally those assumpti<strong>on</strong>s require testing and<br />

c<strong>on</strong>firmati<strong>on</strong>. Yet, care should be taken when adding specific variables with specific local trends, as<br />

these normally add n<strong>on</strong>-stochastic (deterministic) trends in the residuals.<br />

References<br />

Anselin, L. (1988). Spatial Ec<strong>on</strong>ometrics and Methods, Kluwer <strong>Academic</strong> Publishers.<br />

Clough, C. (2010). “Geolearners: Locati<strong>on</strong>-Based Informal Learning with Mobile and Social Technologies”. IEEE<br />

Transacti<strong>on</strong>s <strong>on</strong> Learning Technologies, Vol. 3, No. 1, pp 33-44.<br />

EC – <str<strong>on</strong>g>European</str<strong>on</strong>g> Commissi<strong>on</strong> (2012). Natura 2000 network.<br />

http://ec.europa.eu/envir<strong>on</strong>ment/nature/natura2000/index_en.htm<br />

Ferrão, J. (2002). "Portugal, três geografias em recompilação: Espacialidades, mapas cognitivos e identidades<br />

territoriais”. Lusotopie, Vol. 2, pp.151-158.<br />

280


Teresa Santos et al.<br />

Finglet<strong>on</strong>, B. (1999). "Spurious Spatial Regressi<strong>on</strong>: Some M<strong>on</strong>te Carlo Results with a Spatial Unit Root and<br />

Spatial Cointegrati<strong>on</strong>". Journal of Regi<strong>on</strong>al Science, Vol. 39, No.1, pp.1-19.<br />

Freedman, D.A. (2001). “Ecological Inference and the Ecological Fallacy”. In Neil J. Smelser and Paul B. Baltes<br />

(eds), Internati<strong>on</strong>al Encyclopedia of the Social & Behavioral Sciences. Vol.6, pp. 4027-4030.<br />

Getis, A. (2007). "Reflecti<strong>on</strong>s <strong>on</strong> spatial autocorrelati<strong>on</strong>". Regi<strong>on</strong>al Science and Urban Ec<strong>on</strong>omics, Vol.37,<br />

pp.491-496.<br />

Gelman, A, (2007). “Scaling regressi<strong>on</strong> inputs by dividing by two standard deviati<strong>on</strong>s”. Statistics in Medicine, Vol.<br />

27, Wiley Online Library, pp: 2865-2873.<br />

Gram-Hansen, L.B. (2009). “Geocaching in a persuasive perspective.” Proceedings of the 4th Internati<strong>on</strong>al<br />

<str<strong>on</strong>g>C<strong>on</strong>ference</str<strong>on</strong>g> <strong>on</strong> Persuasive Technology, April 26-29, Clarem<strong>on</strong>t, California, USA.<br />

Hawley, F.F. (2010). “Ag<strong>on</strong> and Ecstasy: Transgressi<strong>on</strong>, Transformati<strong>on</strong>, and Transcendence in Competitive<br />

Geocaching”. Deviant Behavior, Vol. 31, No. 3, pp 225-250.<br />

IGP – Instituto Geográfico Português. (2011). Cartografia de Ocupação do Solo de Portugal C<strong>on</strong>tinental para<br />

2007, [<strong>on</strong>line] http://www.igeo.pt/produtos/CEGIG/Cos2007.htm.<br />

INE - Instituto Naci<strong>on</strong>al de Estatística, I.P (2011). "Censos 2011 – Resultados Provisórios". INE, Lisboa.<br />

Lloyd, C. R. (2011). Local Models for Spatial Analysis, 2nd Editi<strong>on</strong>. CRC Press, Boca Rat<strong>on</strong>.<br />

M<strong>on</strong>z, C.A., Cole, D. N., Leung, Y.F., Mari<strong>on</strong>, J.L. (2010). “Sustaining visitor use in protected areas: Future<br />

opportunities in recreati<strong>on</strong> ecology research based <strong>on</strong> the USA experience”. Envir<strong>on</strong>mental <strong>Management</strong>,<br />

Vol. 45, pp 551-562.<br />

Newsome, D., Lacroix, C., Pickering, C. (2011). “Adventure Racing Events in Australia: c<strong>on</strong>text, assessment and<br />

implicati<strong>on</strong>s for protected area management”, Australian Geographer, Vol. 42, No.4, pp 403-418.<br />

O’Hara, K. (2008). “Understanding Geocaching Practices and Motivati<strong>on</strong>s”. CHI 2008 Proceedings On the Move,<br />

April 5-10, Florence, Italy.<br />

Olaya, V. (2012). Sistemas de Información Geográfica. (http://sextante.googlecode.com/files/Libro_SIG.pdf)<br />

Rodrigues, A. (2001). "The State, Market and Accessibility Triangle: Insights Into the Manufacturing Landscape<br />

of Northern Portugal". PhD. Thesis, Faculty of Urban and Regi<strong>on</strong>al Studies, University of Reading.<br />

Rogers<strong>on</strong>, P. (2001). Statistical Methods for Geography. SAGE Publicati<strong>on</strong>s.<br />

281


Intelligent Decisi<strong>on</strong> Support Systems Development Based<br />

<strong>on</strong> Modern Modeling Methods<br />

Elena Serova<br />

St. Petersburg State University of Ec<strong>on</strong>omics and Finance, St. Petersburg,<br />

Russia<br />

serovah@gmail.com<br />

Abstract: Agent based modeling (ABM) is a new modeling paradigm and <strong>on</strong>e of the most advanced practical<br />

developments in modeling. ABM promises to have far-reaching effects <strong>on</strong> the way that business practiti<strong>on</strong>ers and<br />

academic researchers use informati<strong>on</strong> communicati<strong>on</strong> technologies to support decisi<strong>on</strong> making at different levels<br />

of management. Modern design models and architectural structures are opening up new possibilities and new<br />

applicati<strong>on</strong> areas are coming to the foreground. Multi-agent systems as systems of distributed artificial<br />

intelligence are now having a significant influence <strong>on</strong> informati<strong>on</strong> systems design, simulati<strong>on</strong> and analysis. This<br />

paper focuses <strong>on</strong> the various modeling methods and technologies that are employed in the development of<br />

intelligent decisi<strong>on</strong> support systems. Its goal is to evaluate the role of the agent based modeling in the design of<br />

management decisi<strong>on</strong> processes. The paper c<strong>on</strong>siders the main features of intellectual agent modeling<br />

methodology, and discusses the different types modelling categorizati<strong>on</strong>. It does so from research base that<br />

draws from theoretical underpinnings as well as internati<strong>on</strong>al and domestic industry practices. The basic<br />

principles of agent-based modeling are first introduced and areas of applicati<strong>on</strong> are then discussed from<br />

perspective of real-world applicati<strong>on</strong>s: flow simulati<strong>on</strong>, organizati<strong>on</strong>al simulati<strong>on</strong>, market simulati<strong>on</strong>, and diffusi<strong>on</strong><br />

simulati<strong>on</strong>. The classificati<strong>on</strong> of modeling types is discussed, together with and business applicati<strong>on</strong> simulati<strong>on</strong><br />

frameworks.<br />

Keywords: modeling, management, informati<strong>on</strong> systems, decisi<strong>on</strong> support systems, intellectual agent, multiagent<br />

systems<br />

1. Introducti<strong>on</strong><br />

The use of modern modeling methods and technologies are now essential comp<strong>on</strong>ents for developing<br />

management decisi<strong>on</strong> process that will enable companies to succeed in a rapidly changing<br />

envir<strong>on</strong>ment. It is noteworthy that simulati<strong>on</strong> modeling is now c<strong>on</strong>sidered an essential feature of<br />

decisi<strong>on</strong> making in companies that actively employ modern informati<strong>on</strong> technologies.<br />

Modern modeling tools should facilitate mutual understanding at different organizati<strong>on</strong>al levels when<br />

making strategic management decisi<strong>on</strong>s thus bridging the gaps between a strategic visi<strong>on</strong> and its<br />

implementati<strong>on</strong>. On multi-agent systems (MAS) which, as a class, have developed rapidly over the<br />

last decade. The advantage of a multi-agent approach relates to the ec<strong>on</strong>omic mechanisms of selforganizati<strong>on</strong><br />

and evoluti<strong>on</strong> that become powerful efficiency drivers and c<strong>on</strong>tribute to enterprise’s<br />

development and prosperity. New intellectual data analysis can be created, through MAS which is<br />

open aimed at flexibly adaptive problems solving, and deeply integrated in decisi<strong>on</strong> support systems.<br />

Modern business simulati<strong>on</strong> modeling tools use special software, programming languages and<br />

systems to develop models of business processes, relati<strong>on</strong>s between people and areas for<br />

optimizati<strong>on</strong> in the organizati<strong>on</strong>al structure as a whole.<br />

2. Classificati<strong>on</strong> of modeling types<br />

Modelling is widespread as a means to represent reality. Establishing a classificati<strong>on</strong> of all possible<br />

types of modeling is difficult since the noti<strong>on</strong> of a "model" is used broadly in science and technology,<br />

art, and in everyday life. It is n<strong>on</strong>etheless possibly to distinguish the following types of modeling:<br />

C<strong>on</strong>ceptual;<br />

Physical;<br />

Structured-functi<strong>on</strong>al;<br />

Mathematical and<br />

Simulati<strong>on</strong>.<br />

All these types of modeling can be employed to study complex systems simultaneously, or in certain<br />

combinati<strong>on</strong>s. Traditi<strong>on</strong>ally computer modelling or computer simulati<strong>on</strong> falls within the domain of<br />

simulati<strong>on</strong> and is c<strong>on</strong>cerned with the analysis or syntheses of complex systems in order to support<br />

problem or decisi<strong>on</strong> analysis activities. The focus of computer modeling can include: the ec<strong>on</strong>omic<br />

282


Elena Serova<br />

activity of a company or bank, industrial enterprise, data-processing network, technological process or<br />

any real object or process.<br />

Computer or simulati<strong>on</strong> models of management informati<strong>on</strong> systems display all major factors and<br />

correlati<strong>on</strong>s characterizing real situati<strong>on</strong>s, criteria and limitati<strong>on</strong>s. Models should be universal enough<br />

to describe the phenomena in questi<strong>on</strong>, simple enough to permit research at reas<strong>on</strong>able cost, and<br />

achieve the following objectives:<br />

Reduce the number of functi<strong>on</strong>al roles and management levels, and specifically mid-level<br />

workers;<br />

Rati<strong>on</strong>alize soluti<strong>on</strong>s to management problems by implementing mathematical methods of data<br />

processing, using simulati<strong>on</strong> and artificial intelligence systems;<br />

Create a modern, dynamic organizati<strong>on</strong>al structure, improving enterprise’s flexibility and<br />

manageability;<br />

Reduce administrative costs;<br />

Reduce time spent to planning activities and decisi<strong>on</strong> making;<br />

Increase competitive advantage.<br />

To clarify the role of computer simulati<strong>on</strong> in modern management the structure functi<strong>on</strong> approach to<br />

solving business problems should be noted. The essence of computer modeling in business is to<br />

obtain quantitative and qualitative results from the existing model. Qualitative results allow discovery<br />

of previously unknown features of a complex system including issues such as structure, development<br />

trends, sustainability, integrity, and so <strong>on</strong>. Most quantitative results help forecast certain future values<br />

of variables that characterize the system being modeled.<br />

The essential difference between computer simulati<strong>on</strong> and structure-functi<strong>on</strong> analysis is that the<br />

former yields both qualitative and quantitative results (Serova, 2009).<br />

Another well-known applicati<strong>on</strong> of computer modelling is aimed at solving management problems<br />

through mathematics and logic and, as a rule employs Excel spread sheets. Problems susceptible to<br />

this approach include stock management as well as transport, industrial and marketing logistics<br />

(Gorshkov et al., 2004). The same is possible with problems of linear and multiple regressi<strong>on</strong><br />

forecasting, resource utilizati<strong>on</strong> review, and so <strong>on</strong>.<br />

The Computer model used for managerial decisi<strong>on</strong> making must, as far as possible, encompass the<br />

main factors and interrelati<strong>on</strong>s that characterize real situati<strong>on</strong>s, and their parameters. The model must<br />

be both sufficiently broad so as to include the specificities of management objects and ec<strong>on</strong>omical in<br />

its acti<strong>on</strong>s. With this in mind the following are recommended as appropriate areas for exploiting the<br />

benefits of computer simulati<strong>on</strong>:<br />

Where there are incomplete or incorrect formulati<strong>on</strong>s of the managerial issues involved and the<br />

modelling process involves apprehending the nature of the object to be modelled. Simulati<strong>on</strong> in<br />

this case is used in order to study of the phenomena;<br />

Where analytical methods exist, but mathematical procedures are so complex and labourintensive<br />

that simulati<strong>on</strong> is the simplest path to decisi<strong>on</strong> making;<br />

When observati<strong>on</strong> for behaviour of managerial system’s comp<strong>on</strong>ents is required;<br />

Where simulati<strong>on</strong> is the <strong>on</strong>ly way to study a managerial system because it is impossibility to<br />

observe the target phenomena in real c<strong>on</strong>diti<strong>on</strong>;<br />

Where new situati<strong>on</strong>s are studied in complex systems that are relatively unknown to researches.<br />

In this case simulati<strong>on</strong> is used for preliminary checks <strong>on</strong> a new strategy and decisi<strong>on</strong> rules before<br />

undertaking experiments <strong>on</strong> real systems;<br />

Where the model is used for predicti<strong>on</strong> of bottlenecks in management systems and other<br />

obstacles, which may appear as a result of the introducing new comp<strong>on</strong>ents.<br />

It is obvious, that the modeling methods listed above (c<strong>on</strong>ceptual, simulati<strong>on</strong>, mathematical logic and<br />

structure-functi<strong>on</strong>) —are not mutually exclusive and that they can be applied to management systems<br />

research simultaneously or in combinati<strong>on</strong>.<br />

283


Elena Serova<br />

3. Structure-functi<strong>on</strong>al approach for decisi<strong>on</strong> support<br />

The most intuitive and popular example of the structure-functi<strong>on</strong> computer modeling in modern<br />

management is the business process modeling.<br />

Modern management benefits from business-process improvement in that it obtains a comprehensive<br />

view of the way a company c<strong>on</strong>ducts business; managers should thus know how to c<strong>on</strong>duct process<br />

simulati<strong>on</strong> and analysis using the capabilities of modern software packages and platforms.<br />

The market situati<strong>on</strong> most modern companies operate in is quite unstable obliging them to resp<strong>on</strong>d to<br />

change quickly and accurately. So<strong>on</strong>er or later, businesses must adapt and restructure, and<br />

managers will rethink business processes in order to improve the enterprise’s operati<strong>on</strong>s. Thus, a<br />

manufacturer may wish to rec<strong>on</strong>sider purchasing, ordering or delivery. Business process<br />

reengineering is tied to alterati<strong>on</strong>s in the architecture of informati<strong>on</strong> systems. The key to success with<br />

a reengineering project is close cooperati<strong>on</strong> with of all the groups interested in solving the problem,<br />

primarily IT specialists and experts in the business area. This is achieved by building structurefuncti<strong>on</strong><br />

computer models that reflect business processes which are understandable for all<br />

participants. Such models should simultaneously help formalize the current state of affairs, and find<br />

room for improvement. There are several computer technologies aimed at automating such structure<br />

models—the CASE (Computer Aided Software Engineering) tools which involve various utilities for<br />

analysis and modelling that, together represent just a small fracti<strong>on</strong> of the larger class. Organizati<strong>on</strong><br />

and structure changes in a company involve serious risks especially when they involve the<br />

implementati<strong>on</strong> of an Enterprise Resource Planning System (ERP). The implicati<strong>on</strong>s of such changes<br />

should be carefully studied and analysed before beginning a project. ERPs such as SAP ERP ECC,<br />

BAAN, ROSS iRenaissance, and related use methods and tools that are time tested, minimize risks<br />

and resolve issues that arise from the reorganizati<strong>on</strong> of business processes, including those linked to<br />

the implementati<strong>on</strong> of modern IT systems. Today’s approach to business process design suggests<br />

c<strong>on</strong>tinuous improvement and modificati<strong>on</strong>, analysis and prognosis, as well as timely changes to the<br />

business model. The diagnosis should adequately reflect the current state of affairs to lay a<br />

comprehensive foundati<strong>on</strong> under business development strategy and business automati<strong>on</strong>. There are<br />

steps are recommended for business development or modificati<strong>on</strong> (Figure 1)<br />

Figure 1: Main steps for business development (or modificati<strong>on</strong>)<br />

There are several techniques used when modeling business processes with the most popular being<br />

Business Process Modeling, Work Flow Modeling and Data Flow Modeling (Ananiev, Serova, 2008).<br />

284


Elena Serova<br />

First discussed in the 1970s by Douglas Ross, the Structured Analysis and Design Technique (SADT)<br />

is a foundati<strong>on</strong> for the IDEF0 business process modeling standard (FIPS 183, 1993). The AllFusi<strong>on</strong><br />

Process Modeler 4.1 (aka BPwin 4.1) introduced by Computer Associates (CA) is a modelling tool<br />

that is fully compliant with IDEF0 and allows analysing, documenting and planning changes in<br />

complex business processes scenarios (Maklakov, 2003).<br />

Another actively used process descripti<strong>on</strong> methodology is the Work Flow Modeling applicati<strong>on</strong> — the<br />

IDEF3 standard for building process models as job time sequences of jobs (functi<strong>on</strong>s, operati<strong>on</strong>s).<br />

The ARIS source envir<strong>on</strong>ment provided by IDS Scheer AG which creates methodological and work<br />

instructi<strong>on</strong>s with eEPS (extend Event-driven Process Chain) models, is based <strong>on</strong> IDEF3.<br />

DFD (Data Flow Diagramming) notati<strong>on</strong>s allow <strong>on</strong>e to portray job sequences within a process and the<br />

informati<strong>on</strong> flows circulating between different jobs processes. The DFD methodology minimizes the<br />

subjectivity of business process analysis and can be efficient when implementing a process approach<br />

to organizati<strong>on</strong>al management.<br />

The developing UML (Unified Modeling Language) methodology is also widely used. It embraces a<br />

series of diagrams (e.g., the Activity Diagram) that can be used to describe business processes, even<br />

though business modeling is not what UML’s primary objective.<br />

Al<strong>on</strong>g with the techniques listed above, there are others offered by various software developers.<br />

Corporati<strong>on</strong>s as IBM and Oracle offer their own business process descripti<strong>on</strong> and modeling tools for<br />

example Oracle’s Workflow technology which is used to automate job flows features tools for process<br />

descripti<strong>on</strong> and formalizati<strong>on</strong>. The most popular state-of-the-art business process management<br />

standard is BPEL (Business-Process Executi<strong>on</strong> Language) which allows for the creati<strong>on</strong> of an integral<br />

platform for all applicati<strong>on</strong>s. Public and private instituti<strong>on</strong>s throughout the world are switching to<br />

BPEL. Certain pilot projects have been carried out in Russia which successfully solved IT<br />

infrastructure optimizati<strong>on</strong> problems.<br />

4. The simulati<strong>on</strong> modeling business applicati<strong>on</strong> framework<br />

The major approaches (or methods) in simulati<strong>on</strong> modeling are: System Dynamics (SD), Discrete<br />

Event (DE) and Agent Based (AB). While SD and DE are traditi<strong>on</strong>al approaches, AB is relatively new.<br />

The Dynamic Systems (DS) also exists, but as a rule used to model and design “physical” systems.<br />

If <strong>on</strong>e c<strong>on</strong>siders the levels of abstracti<strong>on</strong> of these methods, Dynamic Systems or “physical” modeling<br />

is situated at the low level. System Dynamics dealing with aggregates is located at the highest level,<br />

and Discrete Event modeling is employed at an intermediate level abstracti<strong>on</strong>. Agent Based modeling,<br />

is used across all levels of abstracti<strong>on</strong>. Agents may model objects of very diverse nature and scale:<br />

at lower levels, for example, pedestrians or cars or robots can be modeled; customer – at<br />

intermediate level, competing companies at the highest level. (Figure 2).<br />

Methods Attributes Abstracti<strong>on</strong><br />

Level (s)<br />

System<br />

Dynamics<br />

(SD)<br />

Agent-<br />

Based<br />

Modeling<br />

(AB)<br />

Aggregates,<br />

Stock-and-Flow<br />

diagrams,<br />

Feedback<br />

loops<br />

Active objects,<br />

Individual<br />

behavior rules,<br />

Direct or<br />

indirect<br />

interacti<strong>on</strong>,<br />

Envir<strong>on</strong>mental<br />

model<br />

High<br />

(minimum<br />

details,<br />

macro level)<br />

High<br />

Middle<br />

Low<br />

<strong>Management</strong><br />

Level (s)<br />

Areas of Applicati<strong>on</strong> Simulati<strong>on</strong><br />

Modeling<br />

Strategic Populati<strong>on</strong> Dynamics,<br />

Ecosystems,<br />

etc.<br />

Strategic<br />

Tactical<br />

Operati<strong>on</strong>al<br />

285<br />

Logistics,<br />

Manufacturing,<br />

IT systems/<br />

Telecommunicati<strong>on</strong>,<br />

Business Processes,<br />

Services,<br />

Asset <strong>Management</strong>,<br />

Project <strong>Management</strong>,<br />

Finance,<br />

Market place &<br />

competiti<strong>on</strong>,<br />

HRM,<br />

etc.<br />

Software<br />

VenSim,<br />

PowerSim,<br />

iThink<br />

AnyLogic,<br />

<strong>Academic</strong><br />

tools: Swarm,<br />

RePast,<br />

NetLogo,<br />

ASCAPE


Methods Attributes Abstracti<strong>on</strong><br />

Level (s)<br />

Discrete<br />

Event<br />

Modeling<br />

(DE)<br />

Dynamic<br />

Systems<br />

(DS)<br />

Entities<br />

(passive<br />

objects),<br />

Flowcharts,<br />

Resources<br />

Physical state<br />

variables,<br />

Block diagrams<br />

and/or<br />

algebraicdifferential<br />

equati<strong>on</strong>s<br />

Middle<br />

(medium<br />

details, meso<br />

level)<br />

Low<br />

Low<br />

(maximum<br />

details, micro<br />

level)<br />

Elena Serova<br />

<strong>Management</strong><br />

Level (s)<br />

Tactical<br />

Operati<strong>on</strong>al<br />

Areas of Applicati<strong>on</strong> Simulati<strong>on</strong><br />

Modeling<br />

Business Processes,<br />

Manufacturing,<br />

Services,<br />

Warehouse,<br />

etc.<br />

Operati<strong>on</strong>al Automotive c<strong>on</strong>trol<br />

systems,<br />

Traffic micro level,<br />

etc.<br />

Software<br />

Arena,<br />

GPSS,<br />

ExtendSim,<br />

SimProcess,<br />

AutoMod,<br />

Promodel,<br />

Enterprise<br />

Dynamics…<br />

MATLAB,<br />

LabView,<br />

VisSim…<br />

Figure 2: Simulati<strong>on</strong> modeling business applicati<strong>on</strong> framework (Nati<strong>on</strong>al Simulati<strong>on</strong> Society (Russia)<br />

and author’s own elaborati<strong>on</strong>s)<br />

System Dynamics is “the study of informati<strong>on</strong>-feedback characteristics of industrial activity to show<br />

how organizati<strong>on</strong>al structure, amplificati<strong>on</strong> (in policies), and time delays (in decisi<strong>on</strong>s and acti<strong>on</strong>s)<br />

interact to influence the success of the enterprise” (Forrester, 1961). The range of SD applicati<strong>on</strong>s<br />

includes also urban, social, ecological types of systems. In SD the real-world processes are<br />

represented in terms of stocks (e.g. of material, knowledge, people, m<strong>on</strong>ey), flows between these<br />

stocks, and informati<strong>on</strong> that determines the values of the flows. SD abstracts from single events and<br />

entities and takes an aggregate view c<strong>on</strong>centrating <strong>on</strong> policies. To approach the problem in SD style<br />

<strong>on</strong>e has to describe the system behavior as a number of interacting feedback loops, balancing or<br />

reinforcing. One of the well-known examples of classic SD model is Bass Diffusi<strong>on</strong> Model.<br />

Discrete Event modelling may be c<strong>on</strong>sidered as definiti<strong>on</strong> of a global entity processing algorithm, with<br />

stochastic elements. This modelling approach roots to 1960s when Geoffrey Gord<strong>on</strong> c<strong>on</strong>ceived and<br />

evolved the idea for GPSS (General Purpose Simulati<strong>on</strong> System) and brought about its IBM<br />

implementati<strong>on</strong>s (Gord<strong>on</strong>, 1961). The term Discrete Event modelling or Discrete Event simulati<strong>on</strong> are<br />

comm<strong>on</strong>ly used for the modelling method that represents the system as a process, i.e. a sequence of<br />

operati<strong>on</strong>s being performed over entities such as customers, parts, documents, etc. These processes<br />

typically include delays, usage of resources, and waiting in queues. Each operati<strong>on</strong> is modelled by its<br />

start event and end event, and no changes can take place in the model in between any two discrete<br />

events. The term discrete has been in general use for decades to distinguish this modelling method<br />

from c<strong>on</strong>tinuous time methods, such as SD. With the emergence of Agent Based modelling the term<br />

Discrete Event modelling in its traditi<strong>on</strong>al sense created c<strong>on</strong>fusi<strong>on</strong> since in most agent based models<br />

acti<strong>on</strong>s are also associated with discrete events, but there may be no processes, entities, or<br />

resources.<br />

Compared to SD or DE models, there is no such place in AB model where the global system<br />

behaviour (dynamics) would be defined. Instead, the modeller defines behaviour at individual level,<br />

and global behaviour emerges as a result of many individuals, each following its own behaviour rules,<br />

living together in some envir<strong>on</strong>ment and communicating with each other and with the envir<strong>on</strong>ment<br />

(Borshchev, Filippov, 2006).<br />

5. Agent-based modeling c<strong>on</strong>cept<br />

Agent technologies offer various types of agents, model of their behavior and characteristics, through<br />

a range of architectures and comp<strong>on</strong>ents libraries. The noti<strong>on</strong> “Agent” has developed from the wellknown<br />

c<strong>on</strong>cept of ‘object’ which is an abstracti<strong>on</strong> from a collecti<strong>on</strong> of real-world items with the same<br />

qualities and behavioral rules.<br />

Am<strong>on</strong>g the various classificati<strong>on</strong>s of agents, the most widely known is that of (Kalchenko, 2005):<br />

Intellectual – Mobile – Stati<strong>on</strong>ary.<br />

286


Elena Serova<br />

Agent qualities are determined by their classificati<strong>on</strong>. Intellectual agents have the most<br />

comprehensive set of qualities; their intellectual capacity allows them to build virtual worlds where<br />

they form acti<strong>on</strong> plans. Minimum set of basic characteristics for any agent includes qualities such as<br />

(Gavrilova, Muromtsev, 2007):<br />

Activity – the ability to organize and carry out acti<strong>on</strong>s;<br />

Aut<strong>on</strong>omy (semi-aut<strong>on</strong>omy) – relative independence from the envir<strong>on</strong>ment and a certain “freewill”<br />

given a good supply of behavioural resources;<br />

Sociability – created by the necessity to carry out tasks in cooperati<strong>on</strong> with other agents and<br />

supported by communicati<strong>on</strong> protocols;<br />

Purpose – innate sources of motivati<strong>on</strong>, or more generally – special intenti<strong>on</strong>al characteristics.<br />

This c<strong>on</strong>cept is close to <strong>on</strong>e of the most popular definiti<strong>on</strong>s of agent by Wooldridge – (2002).<br />

In additi<strong>on</strong> to characteristics we can add:<br />

Adaptability – the ability to learn and reas<strong>on</strong>. Agents may possess partial knowledge or inference<br />

mechanisms, as well as specialize knowledge in a subject matter;<br />

Reactivity – functi<strong>on</strong>al percepti<strong>on</strong> of the envir<strong>on</strong>ment and adaptati<strong>on</strong> to changes therein. This<br />

includes basic knowledge, creeds, wishes, commitments and intenti<strong>on</strong>s.<br />

The technologies that have been used to successfully develop agents and multi-agent systems<br />

include (Kalchenko, 2005):<br />

Knowledge-based systems;<br />

Neur<strong>on</strong> networks;<br />

Clustering algorithms;<br />

Fuzzy logic;<br />

Decisi<strong>on</strong> trees;<br />

Bayes’ theorem;<br />

Genetic algorithms;<br />

Natural language processing.<br />

Multi- (or multiple-) agent systems (MAS), or agent-oriented programming represent a step forward<br />

from object-oriented programming (OOP) and integrate the latest advances in the areas of artificial<br />

intelligence, parallel computing and telecommunicati<strong>on</strong>s. Unlike comm<strong>on</strong> objects in OOP, an agent is<br />

an aut<strong>on</strong>omous object which implies that its behavior is dictated by goals, and that it has competence<br />

to achieve them. Agents cannot be called subprograms (or methods in OOP), because they have their<br />

own states and c<strong>on</strong>tinuously work to achieve their goals—much like co-programs that can pass<br />

c<strong>on</strong>trol to <strong>on</strong>e another at any time. Thus, they can <strong>on</strong>ly be offered new tasks, which they may accept<br />

or decline depending <strong>on</strong> whether the task meets their goals and interests. To ensure their aut<strong>on</strong>omy,<br />

agents can react to events, make and rec<strong>on</strong>sider decisi<strong>on</strong>s, and interact with other agents.<br />

As a rule, software implementati<strong>on</strong> of a traditi<strong>on</strong>al system is centralized, has a hierarchical structure<br />

and executes predetermined algorithms. The code clearly states what, when and how to complete an<br />

acti<strong>on</strong>. Multi-Agent System is a self-organizing network of agents (software objects) that work<br />

c<strong>on</strong>tinuously and simultaneously <strong>on</strong> establishing and rec<strong>on</strong>sidering links. This system is<br />

decentralized: every agent is aut<strong>on</strong>omous and strives to achieve its goals. Changing an agent’s goal<br />

makes other agents adapt their behavior and change their links.<br />

Every MAS c<strong>on</strong>sists of the following comp<strong>on</strong>ents:<br />

A set of organizati<strong>on</strong>al units with a subset of agents and objects;<br />

A set of tasks;<br />

An envir<strong>on</strong>ment - a space where agents and objects exist;<br />

A set of relati<strong>on</strong>s between agents;<br />

A set of agent acti<strong>on</strong>s (operati<strong>on</strong>s <strong>on</strong> objects).<br />

287


Elena Serova<br />

There are various approaches to designing Multi-Agent Systems and three levels can be defined:<br />

c<strong>on</strong>ceptual descripti<strong>on</strong>, initial design and detailed design. At the first level <strong>on</strong>e should describe the<br />

organizati<strong>on</strong>al structure, goals, business processes and informati<strong>on</strong> support all of which act as a<br />

foundati<strong>on</strong> for the next level’s <strong>on</strong>tology. On the next two levels these elements form the organizati<strong>on</strong>al<br />

visualizati<strong>on</strong> - the virtual world where agents act using the <strong>on</strong>tology to achieve their goals and carry<br />

out the set of tasks.<br />

Multi-Agent Systems distribute tasks am<strong>on</strong>g agents, each being c<strong>on</strong>sidered a member of a group or<br />

organizati<strong>on</strong>. The distributi<strong>on</strong> of tasks suggests that each member of a group is assigned a role,<br />

resp<strong>on</strong>sibilities and behavioral requirements.<br />

Agent technologies normally use certain agent typologies and models, as well as MAS architectures,<br />

and are based <strong>on</strong> agent libraries and development support tools for different types of Multi-Agent<br />

Systems.<br />

The world’s best known and most widely approaches to Multi-Agent System development are OMG<br />

MASIF (Object <strong>Management</strong> Group), which is based <strong>on</strong> the c<strong>on</strong>cept of mobile agents; specificati<strong>on</strong>s<br />

by FIPA (Foundati<strong>on</strong> for Intelligent Physical Agents) based <strong>on</strong> an agent’s assumed intellectuality; and<br />

standards by the Defence Advanced Research Projects Agency (DARPA) such as C<strong>on</strong>trol of Agent<br />

Based Systems.<br />

FIPA is an organizati<strong>on</strong> that produces software standards specificati<strong>on</strong>s for heterogeneous and<br />

interacting agents and agent based systems in order to promote agent-based technology and the<br />

interoperability with other technologies. FIPA members include such companies as Avaya, Boeing,<br />

Cisco, Siemens, Toshiba, and various universities and public instituti<strong>on</strong>s. FIPA specificati<strong>on</strong>s aim to<br />

ensure interacti<strong>on</strong> between intellectual agents through standardized communicati<strong>on</strong>s and c<strong>on</strong>tent<br />

languages. In additi<strong>on</strong> to general communicati<strong>on</strong>s, FIPA also works <strong>on</strong> <strong>on</strong>tology and negotiati<strong>on</strong><br />

protocols to support interacti<strong>on</strong> in certain applied areas (transportati<strong>on</strong>, manufacturing, multimedia,<br />

and network communicati<strong>on</strong>s).<br />

The OMG MASIF standard seeks to create c<strong>on</strong>diti<strong>on</strong>s for the migrati<strong>on</strong> of mobile agents from <strong>on</strong>e<br />

multi-agent system to another through standardized CORBA IDL interfaces.<br />

DARPA initiated the Knowledge Sharing Effort that divided agent programming languages into syntax,<br />

semantics and pragmatics:<br />

KIF – Knowledge Interchange Format (syntax);<br />

Ontolingua – a language for defining sharable <strong>on</strong>tologies (semantics);<br />

KQML (Knowledge Query and Manipulati<strong>on</strong> Language) – a high-level interacti<strong>on</strong> language<br />

(pragmatics).<br />

An important element for creating multi-agent systems is the Agent Communicati<strong>on</strong> Language. This<br />

determines the types of messages that agents will exchange. Inter-agent communicati<strong>on</strong>s are<br />

developed through ACL, a language of c<strong>on</strong>tent and <strong>on</strong>tology that determines a set of basic c<strong>on</strong>cepts<br />

to use in cooperative messages. Ontology here is syn<strong>on</strong>ymous to the API (Applicati<strong>on</strong> Programming<br />

Interface) c<strong>on</strong>cept and determines a particular interface for intellectual agents.<br />

6. Multi-agent approach to decisi<strong>on</strong> making<br />

As systems of distributed artificial intelligence Multi-agent Systems have the following advantages for<br />

intellectual supporting decisi<strong>on</strong> making:<br />

They speed up task fulfilment through parallelism and reduce the volume of data transmitted by<br />

passing high-level partial soluti<strong>on</strong>s to other agents;<br />

They are flexible since agents of various capacities are used to carry out a task dynamically<br />

cooperatively;<br />

They are reliable given that functi<strong>on</strong>s that <strong>on</strong>e agent is unable to carry out will be passed to other<br />

agents.<br />

The integrati<strong>on</strong> of Multi-agent Systems in a decisi<strong>on</strong> support system can offer the following benefits:<br />

An informati<strong>on</strong> system specifically adapted to enterprise needs;<br />

288


Elena Serova<br />

More flexibility and ability to adapt to the external envir<strong>on</strong>ment, especially under c<strong>on</strong>diti<strong>on</strong>s of<br />

uncertainty;<br />

The ability to search and obtain unorthodox soluti<strong>on</strong>s;<br />

The c<strong>on</strong>firmati<strong>on</strong> of suppositi<strong>on</strong>s that previously lacked informati<strong>on</strong>;<br />

Faster decisi<strong>on</strong>-making when modeling negotiati<strong>on</strong>s;<br />

The ability to find and resolve potential c<strong>on</strong>flicts of interests in both the external and internal<br />

envir<strong>on</strong>ments;<br />

More reliable decisi<strong>on</strong>s owing to the ability to pass functi<strong>on</strong>s from <strong>on</strong>e agent to another and<br />

redistribute resp<strong>on</strong>sibilities, which is not always possible in real life;<br />

Optimized access to informati<strong>on</strong> for all employees.<br />

A significant advantage of the Multi-agent Systems approach relates to the ec<strong>on</strong>omic mechanisms of<br />

self-organizati<strong>on</strong> and evoluti<strong>on</strong> which become powerful efficiency drivers for the development and<br />

success of an enterprise. The multi-agent approach allows the creati<strong>on</strong> of a new intellectual data<br />

analysis which can be open, flexible, and adaptive, and deeply integrated with other systems.<br />

Experienced-based accounts c<strong>on</strong>cerning MAS applicati<strong>on</strong>s point to the following areas of applicati<strong>on</strong>:<br />

Distributed or network enterprise management;<br />

Complex and multi-functi<strong>on</strong>al logistics;<br />

Virtual organizati<strong>on</strong>s and Internet portals that sell products and services;<br />

<strong>Academic</strong> management of distance-learning systems;<br />

Companies with developed distributi<strong>on</strong> and transportati<strong>on</strong> networks (e.g., Procter & Gamble);<br />

Distributi<strong>on</strong> channels management;<br />

Users’ preferences simulati<strong>on</strong> modelling (e.g., Ford Motor Company).<br />

Large companies realize a number of advantages with the Multi-Agent approach including: faster<br />

problem solving, less data transmissi<strong>on</strong> (since high-level partial soluti<strong>on</strong>s are passed to other agents),<br />

and faster agreements and order placements.<br />

Distributed companies find advantages in improved supply, supervisi<strong>on</strong> and coordinati<strong>on</strong> of remote<br />

divisi<strong>on</strong>s and structures. Companies with a wide and variable products range can react flexibly to<br />

changing c<strong>on</strong>sumer preferences and forecast changes. Service companies retain client interacti<strong>on</strong><br />

scenarios vis-a-vis problem soluti<strong>on</strong>s with MAS technologies.<br />

7. C<strong>on</strong>clusi<strong>on</strong><br />

The increasing demand for optimisati<strong>on</strong> of decisi<strong>on</strong> support systems development has caused leading<br />

modelers to c<strong>on</strong>sider Agent Based modeling and combined approaches in order to obtain deeper<br />

insights into complex and interdependent processes.<br />

Multi-agent systems - as systems of distributed artificial intelligence - herald an era of networked<br />

organizati<strong>on</strong>s that are supported by the interacti<strong>on</strong> of intellectual robots. This facilitates the shift from<br />

powerful centralized systems to fully decentralized <strong>on</strong>es, with hierarchical structure being replaced by<br />

a networked organizati<strong>on</strong>. Rigid bureaucratic “from top to bottom” management is displaced by<br />

negotiati<strong>on</strong>, and planning with flexible arrangements. As a result, producti<strong>on</strong> volumes, profitability,<br />

competitiveness and mobility are growing. A significant advantage of the Multi-Agent System<br />

approach relates to the ec<strong>on</strong>omic mechanisms of self-organizati<strong>on</strong> and evoluti<strong>on</strong> which become<br />

powerful efficiency drivers for development and success of an enterprise. The Multi-Agent approach<br />

allows the creati<strong>on</strong> of new intellectual data analysis which can be open, flexible and adaptive, and<br />

deeply integrated with other systems.<br />

This does not mean however that Agent Based modeling is a replacement for System Dynamics or<br />

Discrete Event modeling. There are many applicati<strong>on</strong>s where SD or DE models can efficiently solve<br />

the problems. If the problem’s requirements fit well with Discrete Event or System Dynamics<br />

modelling paradigms – using these traditi<strong>on</strong>al approaches is more appropriate. In cases where the<br />

289


Elena Serova<br />

system c<strong>on</strong>tains objects with timing, event ordering or other kinds of individual and aut<strong>on</strong>omous<br />

behaviour, then applying Agent Based or mixed approaches is more efficient.<br />

Acknowledgements<br />

The author gratefully acknowledges the c<strong>on</strong>tributi<strong>on</strong> of colleagues from XJ Technology and the<br />

opportunity of study and work with AnyLogic – multi-method simulati<strong>on</strong> tool.<br />

References<br />

Ananiev,I and Serova,E.( 2008) “The Areas of IDEFO Notati<strong>on</strong> Effective Applicati<strong>on</strong> for Tasks of Business-<br />

Processes Descripti<strong>on</strong>”, Vestnik of St. Petersburg State University, <strong>Management</strong> Seria, № 2,St. Petersburg.<br />

Borshchev, A. and Filippov, A. (2004) AnyLogic — “Multi-Paradigm Simulati<strong>on</strong> for Business, Engineering and<br />

Research”, Paper read at the <str<strong>on</strong>g>6th</str<strong>on</strong>g> IIE Annual Simulati<strong>on</strong> Soluti<strong>on</strong>s <str<strong>on</strong>g>C<strong>on</strong>ference</str<strong>on</strong>g>, Orlando, Florida, USA.<br />

Borshchev, A. and Filippov, A. (2006) “From System Dynamics and Discrete Event to Practical Agent Based<br />

Modeling”, [<strong>on</strong>line], XJ Technologies, www.anylogic.com.<br />

FIPS 183 (1993) Integrati<strong>on</strong> definiti<strong>on</strong> for functi<strong>on</strong> modeling (IDEF0), USA.<br />

Forrester, J. (1961) Industrial Dynamics, Cambridge, MA: MIT Press.<br />

Gavrilova, T.A. and Muromtsev, D.I. (2007) Intellectual technologies in management: tools and systems, GSOM<br />

SPbSU, St. Petersburg.<br />

Gord<strong>on</strong>, G. (1961) “A general Purpose Systems Simulati<strong>on</strong> Program”, McMillan NY, Proceedings of EJCC,<br />

Washingt<strong>on</strong>, pp 87-104.<br />

Gorshkov A.F., Evteev B.V., Korshunov V.A.(2004) Computer modeling of management, Exam, Moscow.<br />

Gorodetsky, V.I., Karsaev, O.V., K<strong>on</strong>yushy, O.V.and Samoylov, V.V. (2008) “MAS-based simulati<strong>on</strong> modeling for<br />

flight traffic management”, MSTU CA Scientific Journal, series “Navigati<strong>on</strong> and Flight Traffic <strong>Management</strong>”.<br />

Kalchenko, D. (2005) “Agents are coming to help” ComputerPress, [<strong>on</strong>line], www.compress.ru.<br />

Karpov, Yu.(2005) System simulati<strong>on</strong> modeling. Introducti<strong>on</strong> to modeling with AnyLogic, St. Petersburg: BHV, St.<br />

Petersburg.<br />

Katkalo, V.S. (2006) Strategic management theory evoluti<strong>on</strong>, St. Petersburg University Publishing House, St.<br />

Petersburg.<br />

Kazantsev, A.K.,Serova, Ye.G., Serova, L.S. and Rudenko, Ye.A. (2007) Informati<strong>on</strong> technology resources of<br />

Russia’s ec<strong>on</strong>omy, St. Petersburg University Publishing House, St. Petersburg.<br />

Kouchtch, S.P. (2002) Network approach in marketing: Russia’s experience, SpbSU Vestnik, series<br />

“<strong>Management</strong>”, Vol. 1, No. 8, pp. 81-107.<br />

Law, A.M.and Kelt<strong>on</strong>, W.D. (2000) Simulati<strong>on</strong> Modeling and Analysis, 3rd ed. McGraw-Hill.<br />

Magedanz, T. “OMG AND FIPA standardizati<strong>on</strong> for agent technology: competiti<strong>on</strong> or c<strong>on</strong>vergence?” CORDIS,<br />

[<strong>on</strong>line], cordis.europa.eu/infowin/acts/analysys/products/thematic/agents/ch2/ch2.htm.<br />

Maklakov S.V. (2003) Modeling business-processes with AllFusi<strong>on</strong> Process Modeler, DIALOGUE MYTHS,<br />

Moscow.<br />

Moshella, D. (2003) Customer-driven IT: how users are shaping technology industry growth, MA: Harvard<br />

Business Press, Bost<strong>on</strong>.<br />

Pidd, M. (2004) Computer Simulati<strong>on</strong> in <strong>Management</strong> Science, 5 th Editi<strong>on</strong>, Wiley.<br />

Pospelov, D.A. (1998) “Multi-agent systems – present and future”. Informati<strong>on</strong> Technologies and Computing<br />

Systems, No. 1, pp.14-21.<br />

Serova, E.G. (2009) The role of Multi-agent Approach in Building Informati<strong>on</strong> Infrastructure for a Modern<br />

Company and Carrying Out <strong>Management</strong> Tasks, Internati<strong>on</strong>al book series “Informati<strong>on</strong> Science and<br />

Computing”, Intelligent Informati<strong>on</strong> and Engineering Systems, Vol.3.<br />

Serova, E.G. (2011) “Enterprise Informati<strong>on</strong> Systems of New Generati<strong>on</strong>”, Electr<strong>on</strong>ic Journal Informati<strong>on</strong><br />

Systems Evaluati<strong>on</strong>, <strong>Academic</strong> Publishing Internati<strong>on</strong>al Ltd, Volume 15, Issue 1, pp 116-126.<br />

Wooldridge, M. (2002) Introducti<strong>on</strong> to MultiAgent Systems, Wiley.<br />

290


Integrating Sustainability Indicators in IT/IS Evaluati<strong>on</strong><br />

Gilbert Silvius<br />

HU University of Applied Sciences, Utrecht, The Netherlands<br />

gilbert.silvius@hu.nl<br />

Abstract: This paper explores the integrati<strong>on</strong> of indicators that reflect the c<strong>on</strong>cepts of sustainability into IT/IS<br />

evaluati<strong>on</strong> methods. It is based <strong>on</strong> the observati<strong>on</strong>s that sustainability is <strong>on</strong>e of the most important challenges of<br />

our time and that IT/IS can make a c<strong>on</strong>tributi<strong>on</strong> to sustainable development. IT/IS evaluati<strong>on</strong> methods should<br />

reflect this c<strong>on</strong>tributi<strong>on</strong> and include criteria for the assessment of sustainability aspects. Based <strong>on</strong> identificati<strong>on</strong><br />

IT/IS evaluati<strong>on</strong> methods and an overview of frameworks for sustainability indicators, an analysis is made of the<br />

inclusi<strong>on</strong> of the indicators and principles of sustainability assessment in IT/IS evaluati<strong>on</strong> methods. The analysis<br />

will c<strong>on</strong>clude that integrating sustainability c<strong>on</strong>siderati<strong>on</strong>s in IT/IS evaluati<strong>on</strong> requires far more than a set of<br />

additi<strong>on</strong>al criteria to be c<strong>on</strong>sidered. Integrating sustainability c<strong>on</strong>siderati<strong>on</strong>s in IT/IS evaluati<strong>on</strong> suggests a far<br />

more holistic and elaborated perspective <strong>on</strong> IT/IS evaluati<strong>on</strong> than the infamous IT productivity paradox that is<br />

dominating the discussi<strong>on</strong> <strong>on</strong> the value of IT/IS still today.<br />

Keywords: sustainability, informati<strong>on</strong> technology, informati<strong>on</strong> systems<br />

1. Introducti<strong>on</strong><br />

Sustainability is recognized by the United Nati<strong>on</strong>s as <strong>on</strong>e of the most important challenges of our time<br />

(Glenn and Gord<strong>on</strong>, 1998). How can we develop prosperity without compromising the life of future<br />

generati<strong>on</strong>s? The pressure <strong>on</strong> companies to broaden its reporting and accountability from ec<strong>on</strong>omic<br />

performance for shareholders, to sustainability performance for all stakeholders has increased<br />

substantially (Visser, 2002). Proactively or reactively, companies are looking for ways to integrate<br />

ideas of sustainability in their marketing, corporate communicati<strong>on</strong>s, annual reports and in their<br />

acti<strong>on</strong>s (Hedstrom et al., 1998; Holliday, 2001).<br />

The growing c<strong>on</strong>cern about sustainability and the preservati<strong>on</strong> of our planet is increasingly being<br />

recognized by the informati<strong>on</strong> technology (IT) and informati<strong>on</strong> systems (IS) disciplines. CIOs identify<br />

‘Green IT’ as an important strategic technology (Thibodeau 2007), but the ‘green’ aspects of IT go<br />

bey<strong>on</strong>d the technology. Given IT’s functi<strong>on</strong>al ability to improve, change and reinvent business<br />

processes, it can also be an important c<strong>on</strong>tributor to more sustainable business practices (Kazlauskas<br />

and Hasan, 2009). However, this ‘Greening by IT’ perspective is not reflected in IT/IS evaluati<strong>on</strong><br />

methods, as these methods tend to focus predominantly <strong>on</strong> an ec<strong>on</strong>omic perspective. Probably<br />

fuelled by the much quoted ‘IT productivity paradox’ (Brynjolfss<strong>on</strong>, 1993), researchers and<br />

practiti<strong>on</strong>ers have been challenged to proof that IT/IS brings ec<strong>on</strong>omic value to the organizati<strong>on</strong>. And<br />

although many evaluati<strong>on</strong> models have been developed that also include other variables than Return<br />

<strong>on</strong> Investment (Renkema and Berghout, 1996), the debate <strong>on</strong> the c<strong>on</strong>tributi<strong>on</strong> of IS seems to be<br />

dominated by the ec<strong>on</strong>omic perspective (Silvius, 2010).<br />

This paper explores the integrati<strong>on</strong> of indicators that reflect the c<strong>on</strong>cepts of sustainability into IT/IS<br />

evaluati<strong>on</strong> methods. The paper will present a brief overview if IT/IS evaluati<strong>on</strong> methods and an<br />

explorati<strong>on</strong> of frameworks for sustainability reporting and evaluati<strong>on</strong>. The paper will then analyse how<br />

these two c<strong>on</strong>cepts, IT/IS evaluati<strong>on</strong> and sustainability, fit, and make a number of observati<strong>on</strong>s <strong>on</strong> the<br />

similarities and differences of the c<strong>on</strong>cepts.<br />

2. IT/IS evaluati<strong>on</strong><br />

Through research and in practice, a substantial number of evaluati<strong>on</strong> methods to assess the<br />

c<strong>on</strong>tributi<strong>on</strong> of IS/IT to business performance was developed. After c<strong>on</strong>sidering over 50 evaluati<strong>on</strong><br />

methods Renkema and Berghout (1996) grouped these methods into four categories: Financial<br />

methods, Multi-criteria methods, Ratio methods and Portfolio methods.<br />

2.1 Financial methods<br />

The Financial methods c<strong>on</strong>sider the valuati<strong>on</strong> of an IT/IS investments as an ec<strong>on</strong>omic issue for which<br />

it is irrelevant whether the investment is in IT or in any other resource. As l<strong>on</strong>g as the effects of the<br />

investment are understood, calculating the value of it is merely a financial technicality (Silvius, 2010).<br />

However, in reality capturing value is not quite that straightforward. Financial valuati<strong>on</strong> methods all<br />

have assumpti<strong>on</strong>s and limitati<strong>on</strong>s. Table 1 provides an overview of these valuati<strong>on</strong> methods.<br />

291


Gilbert Silvius<br />

Table 1: Overview of financial valuati<strong>on</strong> methods (based <strong>on</strong> Silvius, 2010)<br />

Valuati<strong>on</strong> method Qualities Limitati<strong>on</strong>s<br />

Return <strong>on</strong> investment Easy to calculate<br />

Easy to interpret (a simple<br />

percentage)<br />

In line with the financial<br />

administrati<strong>on</strong><br />

Pay-back period Quite easy<br />

Internal Rate of<br />

Return<br />

Discounted Cash<br />

Flow / Net Present<br />

Value<br />

Ec<strong>on</strong>omic Value<br />

Added<br />

Real Opti<strong>on</strong>s<br />

Valuati<strong>on</strong><br />

Intuitively coping with risk<br />

Includes the time-value of m<strong>on</strong>ey<br />

Easy to interpret (a simple<br />

percentage)<br />

Based <strong>on</strong> cash-flows<br />

Includes the time-value of m<strong>on</strong>ey<br />

Based <strong>on</strong> cash-flows<br />

Copes with risk<br />

Includes the opportunity value of<br />

m<strong>on</strong>ey<br />

In line with ‘shareholder value’<br />

Includes optimality and<br />

managerial flexibility in<br />

investments<br />

Game theory Includes market developments<br />

Adds a strategic perspective<br />

Outcome sensitive to amortizati<strong>on</strong> method<br />

Ignores the time-value of m<strong>on</strong>ey<br />

Ignores risk<br />

Ignores part of the revenues<br />

Simplistic, does not determine value<br />

Complex<br />

Not in line with the financial administrati<strong>on</strong><br />

Ignores risk<br />

Multiple outcomes, or n<strong>on</strong>e, possible<br />

Complex<br />

Complex to interpret<br />

Not in line with the financial administrati<strong>on</strong><br />

Not c<strong>on</strong>clusive in case of projects with different<br />

durati<strong>on</strong>s<br />

Value calculati<strong>on</strong> based up<strong>on</strong> <strong>on</strong>e of the other<br />

methods<br />

Not in line with the financial administrati<strong>on</strong><br />

Complex<br />

Complex to interpret<br />

Data often not available<br />

Not in line with the financial administrati<strong>on</strong><br />

Data often not available<br />

Not in line with the financial administrati<strong>on</strong><br />

The limitati<strong>on</strong>s of these financial methods to capture the more qualitative aspects of IT/IS value and<br />

impact led to the development of other methods.<br />

2.2 Multi-criteria methods<br />

Multi-criteria methods are a reacti<strong>on</strong> to the problems of capturing the full value of IT/IS investments in<br />

just financial metrics. These methods aim to identify different relevant aspects of value and risk in<br />

order to enable a thorough discussi<strong>on</strong> and an informed discussi<strong>on</strong> (Frisk, 2007). The most influential<br />

method using multiple criteria is Informati<strong>on</strong> Ec<strong>on</strong>omics (Parker et al., 1988). This method is suited for<br />

evaluating a single project as well as a portfolio of projects.<br />

Informati<strong>on</strong> Ec<strong>on</strong>omics identifies assessment criteria in two domains: business (IT/IS demand) and IT<br />

(IT/IS supply). Criteria in the business domain are: Return in Investment, Strategic Match, Competitive<br />

Advantage, <strong>Management</strong> Informati<strong>on</strong>, Competitive Resp<strong>on</strong>se and Organisati<strong>on</strong>al Risk. Criteria in the<br />

IT domain are: Strategic Informati<strong>on</strong> Systems Architecture, Definiti<strong>on</strong>al Uncertainty, Technical<br />

Uncertainty and Infrastructure Risk. The importance or ‘weight’ of the different criteria may not be<br />

equal. <strong>Management</strong> therefore has to decide up<strong>on</strong> a weight factor for each criteria.<br />

Based up<strong>on</strong> the set of criteria and weight factors each project or investment is given a score <strong>on</strong> all of<br />

the criteria. It is crucially important that the scores are underpinned in this more objective way in order<br />

to create acceptati<strong>on</strong> for the results of the evaluati<strong>on</strong> process.<br />

The results of the evaluati<strong>on</strong> process can be presented in a graphically attractive way. The scores <strong>on</strong><br />

the criteria Return in Investment, Strategic Match, Competitive Advantage, <strong>Management</strong> Informati<strong>on</strong>,<br />

Competitive Resp<strong>on</strong>se and Strategic Informati<strong>on</strong> Systems Architecture, are totalled to a score<br />

representing the ‘value’ of the investment. The scores <strong>on</strong> the criteria Organisati<strong>on</strong>al Risk, Definiti<strong>on</strong>al<br />

Uncertainty, Technical Uncertainty and Infrastructure Risk, add up to a total ‘risk’ score. Combining<br />

the two scores in a two-dimensi<strong>on</strong>al graph provides management with a c<strong>on</strong>cise overview of the<br />

investment portfolio.<br />

2.3 Ratio methods<br />

Different from the financial and multi-criteria methods are ratio methods not aimed at evaluating a<br />

specific investment or project, but at finding the ‘right’ level of total IT/IS costs in an organizati<strong>on</strong>. This<br />

level is expressed as a ratio, e.g. IT costs / total revenue or IT costs / employee. The outcome of<br />

292


Gilbert Silvius<br />

these ratios should be c<strong>on</strong>sidered relative to the same ratios at competitors or for <strong>on</strong>e organizati<strong>on</strong> in<br />

time. Lower or higher scores <strong>on</strong> these ratios than comparable organizati<strong>on</strong>s are not per-se right or<br />

wr<strong>on</strong>g, but should give reas<strong>on</strong> for investigati<strong>on</strong> and discussi<strong>on</strong>.<br />

The most prominent author <strong>on</strong> ratio methods is Paul Strassmann. He developed sophisticated ratios<br />

for specific industries. Based <strong>on</strong> his research he remains sceptical about the value of IT investments.<br />

Strassmann states that “For 55% of U.S. firms the computer budget exceeds their ec<strong>on</strong>omic valueadded.”<br />

and “The "right" level of spending for computers reflects the bureaucratic characteristics of a<br />

firm, not revenue or profits.” (Strassmann, 1997). A limitati<strong>on</strong> to the applicability of the ratio methods,<br />

however, is the availability of data required for the ratios.<br />

2.4 Portfolio methods<br />

In 1981 F. Warren McFarlan suggested to analyze and manage IT/IS investments and projects in<br />

terms of revenues and risks using portfolio theory, as was d<strong>on</strong>e in the financial world (Warren<br />

McFarlan, 1981). ‘Portfolio theory’ referred to the ‘modern portfolio theory’ as developed by Markowitz<br />

(Markowitz, 1952). Although appealing, the use of this insight did not really take off until the Clinger-<br />

Cohen Act. This Act states that the management of IT in US government instituti<strong>on</strong>s ‘must reflect a<br />

Portfolio <strong>Management</strong> approach …… and decisi<strong>on</strong>s to terminate or make additi<strong>on</strong>al investments are<br />

based <strong>on</strong> performance much like an investment broker is measured and rewarded based <strong>on</strong><br />

managing risk and achieving results’.<br />

With its appeal <strong>on</strong> portfolio theory, the Clinger-Cohen Act aimed to bring transparency to IT/IS costs<br />

and benefits. When applying portfolio theory to IT projects however, some issues may occur<br />

regarding the scalability of the investments, the tradability of the investments, the unique character of<br />

some investments, the exchangeability of benefits, the unfamiliarity of project risks, etc. Although the<br />

difference in characteristics between financial investments and IT investments does imply limitati<strong>on</strong>s<br />

to the applicability of portfolio theory, some useful insights could be derived (Van Rossum and Silvius,<br />

2006).<br />

An important insight in portfolio theory is the understanding that the value of an investment will be<br />

influenced by other investments or assets in the portfolio. In other words, investment decisi<strong>on</strong>s are not<br />

taken in isolati<strong>on</strong>. Whereas all other evaluati<strong>on</strong> methods study the value of an investment as an<br />

aut<strong>on</strong>omous value, portfolio methods study value of investments in c<strong>on</strong>juncti<strong>on</strong> to other investments<br />

and assets. An insight that appeals to the comm<strong>on</strong> sense when c<strong>on</strong>sidering architectural aspects.<br />

Portfolio theory also points out the importance of having a structured process in place for the<br />

c<strong>on</strong>tinuous evaluati<strong>on</strong> of the total portfolio of IT/IS investments and projects.<br />

3. IT/IS and sustainability<br />

Societal c<strong>on</strong>cerns about the balance between ec<strong>on</strong>omic growth and social wellbeing has been around<br />

as a political and managerial challenge for over 150 years (Dyllick and Hockerts, 2002). Propelled by<br />

the World Commissi<strong>on</strong> <strong>on</strong> Envir<strong>on</strong>ment and Development (1987) and the 1992 Rio Earth Summit, the<br />

opini<strong>on</strong> that n<strong>on</strong>e of these three goals, ec<strong>on</strong>omic growth, social wellbeing and a wise use of natural<br />

resources, can be reached, without c<strong>on</strong>sidering and effecting the other two, got widely accepted<br />

(Keating, 1993). With this widespread acceptance, sustainable development became <strong>on</strong>e of the most<br />

important challenges of our time.<br />

Sustainability in the c<strong>on</strong>text of sustainable development is defined by the World Commissi<strong>on</strong> <strong>on</strong><br />

Envir<strong>on</strong>ment and Development (1987) as "forms of progress that meet the needs of the present<br />

without compromising the ability of future generati<strong>on</strong>s to meet their needs". This broad definiti<strong>on</strong><br />

emphasizes the aspect of future orientati<strong>on</strong> as a basic element of sustainability. This care for the<br />

future implies a wise use of natural resources and other aspects regarding the envir<strong>on</strong>mental<br />

footprint. However, sustainability requires not just an envir<strong>on</strong>mental “green” perspective, but also a<br />

social <strong>on</strong>e. Elkingt<strong>on</strong> (1997), recognizes this in his ‘triple bottom line’ or ‘Triple-P (People, Planet,<br />

Profit)’ c<strong>on</strong>cept (Figure 1): Sustainability is about the balance or harm<strong>on</strong>y between ec<strong>on</strong>omic<br />

sustainability, social sustainability and envir<strong>on</strong>mental sustainability (Elkingt<strong>on</strong>, 1997).<br />

293


Figure 1: The triple-P c<strong>on</strong>cept of sustainability<br />

Gilbert Silvius<br />

In the debate <strong>on</strong> sustainability and IT/IS, a comm<strong>on</strong>ly used term is ‘Green IT’. As was stated in the<br />

introducti<strong>on</strong>, however, a distincti<strong>on</strong> can be made between ‘Green IT’ and ‘Greening by IT’, more<br />

comm<strong>on</strong>ly referred to as ‘Green IS’ (Kazlauskas and Hasan, 2009; Wats<strong>on</strong> et al., 2010). In this<br />

distincti<strong>on</strong>, Green IT refers to the energy efficient utilizati<strong>on</strong> of IT equipment and Green IS to the use<br />

of IS to enable more sustainable business processes (Boudreau et al., 2008).<br />

The envir<strong>on</strong>mental impact of IT/IS has been subject to discussi<strong>on</strong>. The promise of a decreased<br />

envir<strong>on</strong>mental footprint because of paperless offices and tele-working has been opposed by claims of<br />

increased power c<strong>on</strong>sumpti<strong>on</strong> and hazardous waste because of the use of IT needed to operate the<br />

systems. Because of this diversity of direct and indirect indicators, the overall effect of IS/IT <strong>on</strong><br />

envir<strong>on</strong>mental sustainability is not easy to determine. Plepys (2002) c<strong>on</strong>cludes “The discussi<strong>on</strong> <strong>on</strong><br />

what the role is of IT and, particularly, Internet for sustainability is still going <strong>on</strong> and will hardly reach<br />

any definite c<strong>on</strong>clusi<strong>on</strong>, as the envir<strong>on</strong>mental impacts of the new technologies will depend <strong>on</strong> how<br />

they are used”.<br />

The c<strong>on</strong>cepts of sustainability menti<strong>on</strong>ed above suggest that in the debate <strong>on</strong> the sustainability<br />

aspects of IS/IT, not just the envir<strong>on</strong>mental effects should be c<strong>on</strong>sidered, but also the social effects of<br />

IS use. The United Nati<strong>on</strong>s (UN) c<strong>on</strong>cluded that IS has the ability to be a powerful enabler of social<br />

sustainability and c<strong>on</strong>tributor to development goals, because of its unique characteristics to<br />

dramatically improve communicati<strong>on</strong> and the exchange of informati<strong>on</strong> to strengthen and create new<br />

ec<strong>on</strong>omic and social networks (United Nati<strong>on</strong>s Development Program, 2001).<br />

In some parts of the world, IT/IS is c<strong>on</strong>tributing to revoluti<strong>on</strong>ary changes in business and everyday<br />

life. In other parts of the world, the lives of people have hardly been touched by these innovati<strong>on</strong>s. If<br />

people in developing countries are unable to acquire the capabilities for using IS, they will be<br />

increasingly disadvantaged or excluded from participating in the global informati<strong>on</strong> society. The social<br />

and ec<strong>on</strong>omic potential of these new technologies for development is enormous, but so too are the<br />

risks of exclusi<strong>on</strong> (Mansell, 1999). Ec<strong>on</strong>omic research suggests a positive correlati<strong>on</strong> between the<br />

spread of IS and ec<strong>on</strong>omic growth (Siegel, 2003). IS can c<strong>on</strong>tribute to income generati<strong>on</strong> and<br />

poverty reducti<strong>on</strong>. It enables people and enterprises to capture ec<strong>on</strong>omic opportunities by increasing<br />

process efficiency, promoting participati<strong>on</strong> in expanded ec<strong>on</strong>omic networks, and creating<br />

opportunities for employment.<br />

4. Frameworks of sustainability indicators<br />

Crucial for developing more sustainable business practices is the ability to evaluate the sustainability<br />

aspects of different policies and projects, as well as to m<strong>on</strong>itor progress. Or, as Jain (2005) argues:<br />

"The ability to analyze different alternatives or to assess progress towards sustainability will then<br />

depend <strong>on</strong> establishing measurable entities or metrics used for sustainability". The most frequently<br />

used instruments in this c<strong>on</strong>text are frameworks or sets of sustainable development indicators (SDIs),<br />

both as a way of measuring and evaluating (proposed) acti<strong>on</strong>s, and as a way of communicating this<br />

informati<strong>on</strong> (Bell and Morse, 2003).<br />

294


Gilbert Silvius<br />

Many organizati<strong>on</strong>s have developed frameworks of indicators for this goal. In fact, the literature <strong>on</strong><br />

these models is a veritable jungle of different approaches and numerous case studies (Olss<strong>on</strong> et al,<br />

2004). The Internati<strong>on</strong>al Institute for Sustainable Development (IISD) maintains an <strong>on</strong>line directory of<br />

SDI initiatives. This directory includes more than 600 initiatives at nati<strong>on</strong>al and internati<strong>on</strong>al levels by<br />

governments, n<strong>on</strong>-governmental organizati<strong>on</strong>s (NGOs) and individuals. It can therefore be c<strong>on</strong>cluded<br />

that the use of SDI as an evaluative tool is still very much in its infancy (MacGillivray, 1995, Bell and<br />

Morse, 2003) resulting in more questi<strong>on</strong>s than answers. What should be measured and what could be<br />

excluded? What are the most effective indicators? How should they be organised? And how can the<br />

indicators be communicated?<br />

The following secti<strong>on</strong> gives an overview of some of the most influential frameworks for SDIs.<br />

4.1 Natural step framework<br />

One of the first initiatives to bring scientific principles to the assessment of sustainability was by<br />

Swedish scientist Karl-Henrik Robèrt. Robèrt coordinated a c<strong>on</strong>sensus process to define and<br />

operati<strong>on</strong>alize sustainability. At the core of the process lies a c<strong>on</strong>sensus <strong>on</strong> what is called ‘the Natural<br />

Step framework’. The Natural Step Framework is a holistic framework which helps organizati<strong>on</strong>s to<br />

integrate sustainability principles into their business strategies. It provides a tool for developing a<br />

shared visi<strong>on</strong>, shared identity and shared goals am<strong>on</strong>g departments and al<strong>on</strong>g supply chains.<br />

Foundati<strong>on</strong> of the Natural Step Framework is the principle that a company should try to reduce its<br />

negative impacts <strong>on</strong> the biosphere while enabling humans to fulfil their needs. It stimulates companies<br />

to re-think producti<strong>on</strong> processes and product design and to find innovative alternatives for achieving<br />

their business goals. The framework provides a good basis for both awareness raising as well as<br />

strategy development.<br />

4.2 IISD dashboard of sustainability<br />

The IISD is a Canadian-based, public policy research institute, dedicated to advancing sustainable<br />

development. The IISD developed a sustainability ‘dashboard’ that “illustrates the complex<br />

relati<strong>on</strong>ships am<strong>on</strong>g ec<strong>on</strong>omic, social and envir<strong>on</strong>mental issues” (Internati<strong>on</strong>al Institute for<br />

Sustainable Development, 2012). This Dashboard of Sustainability is intended for decisi<strong>on</strong>-makers<br />

and others interested in sustainable development. It is based <strong>on</strong> the Millennium Development Goals<br />

indicators for developing countries. These indicators help define Poverty Reducti<strong>on</strong> Strategies and<br />

m<strong>on</strong>itor the achievement of the Millennium Development Goals.<br />

4.3 WBCSD measuring impact framework<br />

The World Business Council for Sustainable Development (WBCSD) is an organizati<strong>on</strong> of companies<br />

that joined forces in order to create a sustainable future for business, society and the envir<strong>on</strong>ment.<br />

The WBCSD argues that "sustainable development is good for business and business is good for<br />

sustainable development". This view is supported by some ec<strong>on</strong>omists that state that, c<strong>on</strong>trary to the<br />

popular belief that sustainability requires a trade-off of ec<strong>on</strong>omical and envir<strong>on</strong>mental/social benefits,<br />

it is possible for the c<strong>on</strong>cepts of sustainable development and competitiveness to merge if enacted<br />

wisely (Esty and Porter, 1998).<br />

The WBCSD developed a framework, the Measuring Impact Framework, to assess the c<strong>on</strong>tributi<strong>on</strong> of<br />

business to the ec<strong>on</strong>omic and broader development goals in the societies where business operates. It<br />

is designed to help companies understand their c<strong>on</strong>tributi<strong>on</strong> to society and use this understanding to<br />

inform their operati<strong>on</strong>al and l<strong>on</strong>g-term investment decisi<strong>on</strong>s, and have better-informed c<strong>on</strong>versati<strong>on</strong>s<br />

with stakeholders.<br />

The Measuring Impact Framework includes a 4-step methodology to help companies in any industry<br />

operating in any part of the world to measure, assess and manage their impacts <strong>on</strong> society. In the<br />

applicati<strong>on</strong> of the methodology, an organizati<strong>on</strong> should adapted it to the specific company strategy<br />

and development c<strong>on</strong>text in which the business operates;<br />

4.4 UN global compact framework<br />

The United Nati<strong>on</strong>s (UN) Global Compact (2010) is a framework of ten universally accepted<br />

principles, developed by the UN and a number of large corporati<strong>on</strong>s. It covers the areas of human<br />

rights, labour, envir<strong>on</strong>ment and anti-corrupti<strong>on</strong>. Participating companies agree to comply with these<br />

295


Gilbert Silvius<br />

principles. They can use the framework as a platform for disclosure. This initiative has been created<br />

because the UN realized that businesses are primary drivers for globalizati<strong>on</strong> and can help ensure<br />

l<strong>on</strong>g-term value creati<strong>on</strong> that can bring benefit to ec<strong>on</strong>omies and societies all over the globe. In the<br />

absence of global regulati<strong>on</strong>s, this voluntary code of c<strong>on</strong>duct has been developed, hoping to stimulate<br />

companies to more sustainable business practices.<br />

4.5 UNCSD indicators of sustainable development<br />

Following the 1992 Rio Earth Summit, the UN Commissi<strong>on</strong> <strong>on</strong> Sustainable Development (UNCSD)<br />

started the development of the ‘Indicators of Sustainable Development’. This resulted in a set of 134<br />

indicators of sustainable development. Country case studies and further discussi<strong>on</strong> in the UNCSD led<br />

to the rejecti<strong>on</strong> of a framework in lieu of “themes” and a more comprehensive set of ‘core’ indicators.<br />

The third, revised set of the UNCSD indicators was finalized in 2006 by a group of experts from<br />

developing and developed countries and internati<strong>on</strong>al organizati<strong>on</strong>s. This third editi<strong>on</strong> of the indicator<br />

set is based <strong>on</strong> the previous two (1996 and 2001) editi<strong>on</strong>s, which have been developed, improved<br />

and extensively tested. It c<strong>on</strong>tains 96 indicators, including a subset of 50 core indicators. The<br />

guidelines <strong>on</strong> indicators and their detailed methodology sheets are available as a reference for all<br />

countries to develop nati<strong>on</strong>al indicators of sustainable development.<br />

4.6 ISO 26000 core subjects and issues<br />

As a resp<strong>on</strong>se to businesses’ growing interest and the increasing number of sustainability-related<br />

instituti<strong>on</strong>s and frameworks, the Internati<strong>on</strong>al Organizati<strong>on</strong> for Standardizati<strong>on</strong> (ISO) launched ISO<br />

26000, a comprehensive guideline <strong>on</strong> social resp<strong>on</strong>sibility, to help companies introduce more<br />

sustainable practices. ISO 26000 is a guideline <strong>on</strong> social resp<strong>on</strong>sibility that is designed for all types of<br />

organizati<strong>on</strong>s.ISO 26000 summarizes seven social resp<strong>on</strong>sibility ‘core subjects’: Organizati<strong>on</strong>al<br />

governance, Human rights, Labour practices, The envir<strong>on</strong>ment, Fair operating practices, C<strong>on</strong>sumer<br />

issues and Community involvement and development. These core subjects are further broken down<br />

into ‘issues’, specific themes or activities a company should work <strong>on</strong> in order to c<strong>on</strong>tribute to<br />

sustainable development.<br />

4.7 GRI sustainability reporting guidelines<br />

The Global Reporting Initiative (GRI) is a n<strong>on</strong>-profit organizati<strong>on</strong> that pi<strong>on</strong>eered the world’s most<br />

widely used sustainability reporting framework, the Sustainability Reporting Guidelines (SRG).<br />

Companies can use the SRG to indicate to shareholders and c<strong>on</strong>sumers their ec<strong>on</strong>omic, social and<br />

envir<strong>on</strong>mental performance. GRI’s objective is to facilitate sustainability reporting for companies and<br />

thereby stimulate them to operate more sustainably. The SRG framework c<strong>on</strong>sists of an extensive set<br />

of indicators, from which companies can select a set that is relevant to their operati<strong>on</strong>s or industry.<br />

4.8 Dow J<strong>on</strong>es sustainability indexes<br />

The Dow J<strong>on</strong>es Sustainability Indexes (DJSI) are not a reporting tool, but a family of indexes<br />

evaluating the sustainability performance of the largest 2,500 companies listed <strong>on</strong> the Dow J<strong>on</strong>es.<br />

They are the l<strong>on</strong>gest-running global sustainability benchmarks worldwide. The DJSI is based <strong>on</strong> an<br />

analysis of corporate ec<strong>on</strong>omic, envir<strong>on</strong>mental and social performance, assessing issues such as<br />

corporate governance, risk management, branding, climate change mitigati<strong>on</strong>, supply chain standards<br />

and labor practices. It includes general as well as industry specific sustainability criteria.<br />

From this overview of SDI frameworks it should be c<strong>on</strong>cluded that, although many organizati<strong>on</strong>s have<br />

offered meaningful lists of indicators, c<strong>on</strong>sensus <strong>on</strong> how to measure and assess sustainability has not<br />

emerged yet. A recurring structure in many frameworks is the Triple-P c<strong>on</strong>cept menti<strong>on</strong>ed in secti<strong>on</strong><br />

3. However, some frameworks, for example ISO 26000, adopt a completely different structure and<br />

also different perspectives. Many specialists actually questi<strong>on</strong> whether or not a comm<strong>on</strong> list is even<br />

possible, given the wide variety of c<strong>on</strong>diti<strong>on</strong>s and the differences in values in different c<strong>on</strong>texts.<br />

In the so called Bellagio principles, a set of overarching principles for the assessment of sustainability<br />

are formulated, thereby suggesting that a truly ‘universal’ framework to measure sustainability may be<br />

illusive. The Bellagio principles are (Internati<strong>on</strong>al Institute for Sustainable Development, 1997):<br />

296


Principle 1: Guiding Visi<strong>on</strong> and Goals<br />

Gilbert Silvius<br />

Assessment of progress toward sustainable development should:<br />

Be guided by a clear visi<strong>on</strong> of sustainable development and goals that define that visi<strong>on</strong>.<br />

Principle 2: Holistic Perspective<br />

Assessment of progress toward sustainable development should:<br />

Include review of the whole system as well as its parts.<br />

C<strong>on</strong>sider the well-being of social, ecological, and ec<strong>on</strong>omic sub-systems, their state as well as<br />

the directi<strong>on</strong> and rate of change of that state, of their comp<strong>on</strong>ent parts, and the interacti<strong>on</strong><br />

between parts.<br />

C<strong>on</strong>sider both positive and negative c<strong>on</strong>sequences of human activity, in a way that reflects the<br />

costs and benefits for human and ecological systems, in m<strong>on</strong>etary and n<strong>on</strong>-m<strong>on</strong>etary terms.<br />

Principle 3: Essential Elements<br />

Assessment of progress toward sustainable development should:<br />

C<strong>on</strong>sider equity and disparity within the current populati<strong>on</strong> and between present and future<br />

generati<strong>on</strong>s, dealing with such c<strong>on</strong>cerns as resource use, over-c<strong>on</strong>sumpti<strong>on</strong> and poverty, human<br />

rights, and access to services, as appropriate.<br />

C<strong>on</strong>sider the ecological c<strong>on</strong>diti<strong>on</strong>s <strong>on</strong> which life depends.<br />

C<strong>on</strong>sider ec<strong>on</strong>omic development and other, n<strong>on</strong>-market activities that c<strong>on</strong>tribute to human/social<br />

well-being<br />

Principle 4: Adequate Scope<br />

Assessment of progress toward sustainable development should:<br />

Adopt a time horiz<strong>on</strong> l<strong>on</strong>g enough to capture both human and ecosystem time scales thus<br />

resp<strong>on</strong>ding to needs of future generati<strong>on</strong>s as well as those current to short term decisi<strong>on</strong>-making.<br />

Define the space of study large enough to include not <strong>on</strong>ly local but also l<strong>on</strong>g distance impacts <strong>on</strong><br />

people and ecosystems.<br />

Build <strong>on</strong> historic and current c<strong>on</strong>diti<strong>on</strong>s to anticipate future c<strong>on</strong>diti<strong>on</strong>s.<br />

Principle 5: Practical Focus<br />

Assessment of progress toward sustainable development should be based <strong>on</strong>:<br />

An explicit set of categories or an organizing framework that links visi<strong>on</strong> and goals to indicators<br />

and assessment criteria.<br />

A limited number of key issues for analysis.<br />

A limited number of indicators or indicator combinati<strong>on</strong>s to provide a clearer signal of progress.<br />

Standardizing measurement wherever possible to permit comparis<strong>on</strong>.<br />

Comparing indicator values to targets, reference values, ranges, thresholds, or directi<strong>on</strong> of trends,<br />

as appropriate.<br />

Principle 6: Openness<br />

Assessment of progress toward sustainable development should:<br />

Make the methods and data that are used accessible to all.<br />

Make explicit all judgments, assumpti<strong>on</strong>s, and uncertainties in data and interpretati<strong>on</strong>s.<br />

Principle 7: Effective Communicati<strong>on</strong><br />

Assessment of progress toward sustainable development should:<br />

Be designed to address the needs of the audience and set of users.<br />

297


Gilbert Silvius<br />

Draw from indicators and other tools that are stimulating and serve to engage decisi<strong>on</strong>-makers.<br />

Aim, from the outset, for simplicity in structure and use of clear and plain language.<br />

Principle 8: Broad Participati<strong>on</strong><br />

Assessment of progress toward sustainable development should:<br />

Obtain broad representati<strong>on</strong> of key grass-roots, professi<strong>on</strong>al, technical and social groups,<br />

including youth, women, and indigenous people - to ensure recogniti<strong>on</strong> of diverse and changing<br />

values.<br />

Ensure the participati<strong>on</strong> of decisi<strong>on</strong>-makers to secure a firm link to adopted policies and resulting<br />

acti<strong>on</strong>.<br />

Principle 9: Ongoing Assessment<br />

Assessment of progress toward sustainable development should:<br />

Develop a capacity for repeated measurement to determine trends.<br />

Be iterative, adaptive, and resp<strong>on</strong>sive to change and uncertainty because systems are complex<br />

and change frequently.<br />

Adjust goals, frameworks, and indicators as new insights are gained.<br />

Promote development of collective learning and feedback to decisi<strong>on</strong>-making.<br />

Principle 10: Instituti<strong>on</strong>al Capacity<br />

C<strong>on</strong>tinuity of assessing progress toward sustainable development should be assured by:<br />

Clearly assigning resp<strong>on</strong>sibility and providing <strong>on</strong>going support in the decisi<strong>on</strong>-making process.<br />

Providing instituti<strong>on</strong>al capacity for data collecti<strong>on</strong>, maintenance, and documentati<strong>on</strong>.<br />

Supporting development of local assessment capacity.<br />

These principles provide guidance in the analysis of the impact of integrating sustainability indicators<br />

in IT/IS evaluati<strong>on</strong>, as reported in the next secti<strong>on</strong>.<br />

5. Analysis<br />

When c<strong>on</strong>fr<strong>on</strong>ting the methods of IT/IS evaluati<strong>on</strong> identified in secti<strong>on</strong> 2 with the overarching Bellagio<br />

principles derived from frameworks of sustainability indicators, a number of observati<strong>on</strong>s can be<br />

made.<br />

5.1 Integrating sustainability indicators suggests a multi-criteria approach<br />

The principles Holistic Perspective and Essential Elements, prescribe for sustainability evaluati<strong>on</strong> the<br />

use of more perspectives than just the ec<strong>on</strong>omic perspective. Most frameworks of sustainability<br />

indicators adopt the triple-P c<strong>on</strong>cept and some frameworks take an even more holistic view. This<br />

suggests that by definiti<strong>on</strong> an IT/IS evaluati<strong>on</strong> approach based solely <strong>on</strong> the ec<strong>on</strong>omic perspective is<br />

inadequate for capturing the sustainability aspects of IT/IS.<br />

From the four groups of methods identified by Renkema and Berghout, the multi-criteria group of<br />

methods seems most appropriate to include the multiple perspectives that the c<strong>on</strong>cepts of<br />

sustainability imply.<br />

5.2 Inclusi<strong>on</strong> of sustainability indicators makes sense<br />

Multi-criteria methods for IT/IS evaluati<strong>on</strong>, like Informati<strong>on</strong> Ec<strong>on</strong>omics, typically include an indicator<br />

for the c<strong>on</strong>tributi<strong>on</strong> of IT/IS to the strategy of the organisati<strong>on</strong>. IT/IS evaluati<strong>on</strong> therefore links to<br />

strategy (Silvius, 2010). And as more and more companies are integrating statements about<br />

sustainability in their strategy (Hedstrom et al., 1998; Holliday, 2001), inclusi<strong>on</strong> of sustainability<br />

indicators in IT/IS evaluati<strong>on</strong> makes sense. In the Bellagio perspectives this link is captured in the<br />

principle Guiding Visi<strong>on</strong> and Goals, that prescribes that assessment of sustainability aspects should<br />

be guided by a clear visi<strong>on</strong> of sustainable development and goals that define that visi<strong>on</strong>.<br />

298


Gilbert Silvius<br />

5.3 A universal model for evaluating IT/IS is illusive<br />

The recogniti<strong>on</strong> in the Bellagio principles that a sensible and meaningful set of sustainability indicators<br />

is c<strong>on</strong>text specific, and that c<strong>on</strong>sensus should be sought <strong>on</strong> the level of principles rather than specific<br />

indicators, suggests that a ‘<strong>on</strong>e size fits all’ approach to IT/IS evaluati<strong>on</strong> may not be viable. This is<br />

also recognized by principle Practical Focus that stated that standardizing of measurement should be<br />

sought wherever possible, thereby suggesting that this is not always possible. For organizati<strong>on</strong>s this<br />

would imply that working with a universal ‘business case’ model, which most organizati<strong>on</strong>s do in order<br />

to be able to compare IT/IS investments and projects, actually does not lead to optimal decisi<strong>on</strong><br />

making.<br />

5.4 Including sustainability assessment expands scope<br />

The logical ‘unit of analysis’ of IT/IS evaluati<strong>on</strong> is the organizati<strong>on</strong> that uses the technology or<br />

systems or invests in them. This scope is based <strong>on</strong> ec<strong>on</strong>omical reas<strong>on</strong>ing and the c<strong>on</strong>cept of<br />

ownership. IT/IS should bring benefits to the ec<strong>on</strong>omical unit that invests in, pays for or owns the<br />

technology or systems being assessed. In sustainability assessments, however, the sphere of<br />

influence is not limited to ec<strong>on</strong>omical units or ownership. This is covered in the Bellagio principles<br />

Holistic Perspective and Adequate Scope. The principle Holistic Perspective menti<strong>on</strong>s that<br />

assessment of sustainability should include a review of the whole system as well as its parts. The<br />

principle Adequate Scope prescribes that assessment of sustainability aspects should define the<br />

space of study large enough to include not <strong>on</strong>ly local but also l<strong>on</strong>g distance impacts <strong>on</strong> people and<br />

ecosystems.<br />

5.5 Including sustainability assessment implies equality of time<br />

The ec<strong>on</strong>omic perspective, that is so dominant in all IT/IS evaluati<strong>on</strong> methods, values short term<br />

effects more than l<strong>on</strong>g term effects. This principle is most visible in the discounting of future cash<br />

flows. In ec<strong>on</strong>omic theory an immediately cash flow holds more value than a future cash flow, thereby<br />

emphasizing the value of short-term benefits. However, social impacts or envir<strong>on</strong>mental degradati<strong>on</strong><br />

because of business decisi<strong>on</strong>s, may not occur before the l<strong>on</strong>g-term. Also this aspect is menti<strong>on</strong>ed in<br />

the principle Adequate Scope, that states that assessing sustainability should adopt a time horiz<strong>on</strong><br />

l<strong>on</strong>g enough to capture both human and ecosystem time scales thus resp<strong>on</strong>ding to needs of future<br />

generati<strong>on</strong>s as well as those current to short term decisi<strong>on</strong>-making.<br />

5.6 Sustainability assessment implies c<strong>on</strong>tinuous assessment and instituti<strong>on</strong>al<br />

capacity<br />

The Bellagio principles Ongoing Assessment and Instituti<strong>on</strong>al Capacity prescribe an instituti<strong>on</strong>alized,<br />

repetitive and iterative process to assess sustainability aspects. In IT/IS evaluati<strong>on</strong>, this aspect is<br />

covered in some of the IT/IS evaluati<strong>on</strong> methods, most explicitly in the c<strong>on</strong>cept of portfolio<br />

management. Portfolio management suggests a c<strong>on</strong>tinuous process of m<strong>on</strong>itoring, measuring,<br />

evaluating and selecting investments or assets. In fact, also project management methodologies like<br />

Prince2 include a c<strong>on</strong>tinuous re-assessment of the business case of the project. Assessing<br />

sustainability, however, goes even further than this and suggests that the technology or system at<br />

hand is also c<strong>on</strong>tinuously assessed during its exploitati<strong>on</strong>. This could be compared with the ‘business<br />

case management’ in the post-implementati<strong>on</strong> phase of a project.<br />

5.7 Sustainability assessment implies openness and broad participati<strong>on</strong><br />

The Bellagio principles Openness, Effective Communicati<strong>on</strong> and Broad Participati<strong>on</strong> prescribe how<br />

stakeholders are informed and engaged in the assessment of sustainability aspects. These principles<br />

touch up<strong>on</strong> the way assessments are performed. And although these aspects are not explicitly<br />

covered in the IT/IS evaluati<strong>on</strong> methods, the graphical representati<strong>on</strong>s that are included in the<br />

Informati<strong>on</strong> ec<strong>on</strong>omics methodology do facilitate participati<strong>on</strong> of and communicati<strong>on</strong> with key<br />

stakeholders and decisi<strong>on</strong> makers. It is, however, debatable whether these formats and techniques<br />

are an adequate operati<strong>on</strong>alizati<strong>on</strong> of the principle Openness.<br />

6. C<strong>on</strong>clusi<strong>on</strong><br />

Sustainability is <strong>on</strong>e of the most important challenges of our time. How can we develop prosperity,<br />

without compromising our wellbeing or that of future generati<strong>on</strong>s? More and more companies<br />

recognize this and take resp<strong>on</strong>sibility for their role in this challenge. IT/IS can make a c<strong>on</strong>tributi<strong>on</strong> to<br />

299


Gilbert Silvius<br />

the sustainable development of organisati<strong>on</strong>s. It therefore makes sense to include an assessment of<br />

sustainability aspects in the evaluati<strong>on</strong> of IT/IS. And although some c<strong>on</strong>siderati<strong>on</strong>s of sustainability<br />

can be found in the various methods of IT/IS evaluati<strong>on</strong>, it has to be c<strong>on</strong>cluded that the integrati<strong>on</strong> of<br />

sustainability indicators in IT/IS evaluati<strong>on</strong> is just in its infancy.<br />

In a c<strong>on</strong>tributi<strong>on</strong> to the understanding of sustainability c<strong>on</strong>siderati<strong>on</strong>s in IT/IS evaluati<strong>on</strong>, this paper<br />

c<strong>on</strong>fr<strong>on</strong>ted the principles of sustainability assessment with the different methods of IT/IS evaluati<strong>on</strong>.<br />

This analysis did not result in a set of additi<strong>on</strong>al criteria to be c<strong>on</strong>sidered, but in a set of observati<strong>on</strong>s<br />

that form a foundati<strong>on</strong> to rec<strong>on</strong>sider IT/IS evaluati<strong>on</strong> methods. These observati<strong>on</strong>s are:<br />

Integrating sustainability indicators suggests a multi-criteria approach;<br />

Inclusi<strong>on</strong> of sustainability indicators makes sense;<br />

A universal model for evaluating IT/IS is illusive;<br />

Including sustainability assessment expands scope;<br />

Including sustainability assessment implies equality of time;<br />

Sustainability assessment implies c<strong>on</strong>tinuous assessment and instituti<strong>on</strong>al capacity;<br />

Sustainability assessment implies openness and broad participati<strong>on</strong>.<br />

The implicati<strong>on</strong>s of these observati<strong>on</strong>s may be far fetching, as their impact suggests a far more<br />

holistic and elaborated perspective <strong>on</strong> IT/IS evaluati<strong>on</strong> than the infamous IT productivity paradox. The<br />

operati<strong>on</strong>alizati<strong>on</strong> of this holistic and elaborated evaluati<strong>on</strong> perspective, however, is still subject to<br />

further research.<br />

References<br />

Bell, S and Morse, S. (2003) Measuring Sustainability Learning from doing, Earthscan, L<strong>on</strong>d<strong>on</strong>.<br />

Boudreau, M., Chen, A. and Huber, M. (2008) “Green IS: Building Sustainable Business Practices”, in Wats<strong>on</strong>, R.<br />

(Ed.) Informati<strong>on</strong> Systems: a Global Text project.<br />

Brynjolfss<strong>on</strong>, E. (1993) “The productivity paradox of informati<strong>on</strong> technology”, Communicati<strong>on</strong>s of the ACM,<br />

36(12), pp. 67-77.<br />

Dyllick, T. and Hockerts, K. (2002) “Bey<strong>on</strong>d the business case for corporate sustainability”, Business Strategy<br />

and the Envir<strong>on</strong>ment, 11, pp. 130-141.<br />

Elkingt<strong>on</strong>, J. (1997) Cannibals with Forks: the Triple Bottom Line of 21st Century Business, Capst<strong>on</strong>e Publishing<br />

Ltc. Oxford.<br />

Esty, D. C. and Porter, M. E. (1998) “Industrial Ecology and Competitiveness: Strategic Implicati<strong>on</strong>s for the Firm”,<br />

Journal of Industrial Ecology, 2(1), pp. 35-43.<br />

Frisk, E. (2007) “Categorizati<strong>on</strong> and overview of IT perspectives – A literature review.”, Paper read at the<br />

<str<strong>on</strong>g>European</str<strong>on</strong>g> <str<strong>on</strong>g>C<strong>on</strong>ference</str<strong>on</strong>g> <strong>on</strong> Informati<strong>on</strong> <strong>Management</strong> and Evaluati<strong>on</strong>, M<strong>on</strong>pellier.<br />

Glenn, J. C. and Gord<strong>on</strong>, T. J. (1998) State of the Future: Issues and Opportunities, The Millennium Project,<br />

American Council for the United Nati<strong>on</strong>s University, Washingt<strong>on</strong>, DC.<br />

Hedstrom G, Poltorzycki S, Stroh P. (1998) “Sustainable development: the next generati<strong>on</strong>”, Sustainable<br />

Development: How Real, How So<strong>on</strong>, and Who’s Doing What?, Prism, 4: 5–19.<br />

Holliday C. (2001) “Sustainable growth, the DuP<strong>on</strong>t way”, Harvard Business Review, September, pp.129–134.<br />

Internati<strong>on</strong>al Institute for Sustainable Development (1997) Principles in Practice, Internati<strong>on</strong>al Institute for<br />

Sustainable Development.<br />

Internati<strong>on</strong>al Institute for Sustainable Development (2012) Dashboard for Sustainability,<br />

http://www.iisd.org/cgsdi/dashboard.asp, accessed April 7 th , 2012.<br />

Jain, R. (2005) “Sustainability: metrics, specific indicators and preference index”, Clean Technologies and<br />

Envir<strong>on</strong>mental Policy, May, pp. 71-72.<br />

Kazlauskas, A. and Hasan, H. (2009) “Web 2.0 Soluti<strong>on</strong>s to Wicked Climate Change Problems”, Australiasian<br />

Journal of Informati<strong>on</strong> Systems, 16(2).<br />

Keating, M. (1993) The Earth Summit’s Agenda for Change, Centre for our Comm<strong>on</strong> Future, Geneva.<br />

MacGillivray, A. And Zadek, S. (1995) Accounting for change: indicators for sustainable development, New<br />

Ec<strong>on</strong>omic Foundati<strong>on</strong>, L<strong>on</strong>d<strong>on</strong>.<br />

Mansell, R. (1999) “Informati<strong>on</strong> and communicati<strong>on</strong> technologies for development: assessing the potential and<br />

the risks”, Telecommunicati<strong>on</strong>s Policy, 23, pp. 35-50.<br />

Markowitz, H.M. (1952) “Portfolio selecti<strong>on</strong>”, Journal of Finance, 7 (1).<br />

Olss<strong>on</strong>, J.A., Hilding-Rydevik, T., Aalbu H. and Bradley, K. (2004) “Indicators for Sustainable Development”,<br />

Discussi<strong>on</strong> paper, <str<strong>on</strong>g>European</str<strong>on</strong>g> Regi<strong>on</strong>al Network <strong>on</strong> Sustainable Development.<br />

Parker, M.M., Bens<strong>on</strong>, R.J. and Trainor, H.E. (1988) Informati<strong>on</strong> Ec<strong>on</strong>omics, Linking Business Performance to<br />

Informati<strong>on</strong> Technology, L<strong>on</strong>d<strong>on</strong>: Prentice-Hall.<br />

Plepys, A. (2002) “The grey side of ICT”, Envir<strong>on</strong>mental Impact Assessment Review, 22(5), pp. 509-523.<br />

300


Gilbert Silvius<br />

Renkema T.J.W. and Berghout, E.W. (1996) “Methodologies for informati<strong>on</strong> systems evaluati<strong>on</strong> at the proposal<br />

stage: a comparative review”, Informati<strong>on</strong> and Software Technology, Elsevier.<br />

Rossum, R.B. van, and Silvius, A.J.G. (2006) “ICT Portfolio <strong>Management</strong> in theorie en praktijk; Over slagkracht<br />

en spraakverwarring”, J. van B<strong>on</strong> (Ed.), ITSM Best Practices deel 3, Van Haren Publishing (in Dutch).<br />

Siegel, D. (2003) “ICT, the Internet, and Ec<strong>on</strong>omic Performance: Empirical Evidence and Key Policy Issues”.<br />

UNCTAD and UNECE <str<strong>on</strong>g>C<strong>on</strong>ference</str<strong>on</strong>g>. Geneva.<br />

Silvius, A.J.G. (2010) “A C<strong>on</strong>ceptual Model for Aligning IT Valuati<strong>on</strong> Methods”, Internati<strong>on</strong>al Journal of<br />

IT/Business Alignment and Governance, 1(3), IGI Global, Hershey PA.<br />

Silvius, A.J.G., Schipper, R., Planko, J., Brink, J. van der and Köhler, A. (2012) Sustainability in Project<br />

<strong>Management</strong>, Gower Publishing, Farnham.<br />

Strassmann, P.A. (1997) The Squandered Computer, Informati<strong>on</strong> Ec<strong>on</strong>omic Press.<br />

Thibodeau, P. (2007) “Gartner’s Top 10 Strategic Technologies for 2008”, Computerworld, October 9.<br />

United Nati<strong>on</strong>s Development Program (2001) Creating Value for All: Strategies for Doing Business with the Poor,<br />

Growing Inclusive Markets initiative.<br />

United Nati<strong>on</strong>s Global Compact (2010) UN Global Compact, Retrieved 22-10-2010, from globalcompact:<br />

http://www.unglobalcompact.org/AboutTheGC/index.html.<br />

Visser W.T (2002) “Sustainability reporting in South Africa”, Corporate Envir<strong>on</strong>mental Strategy, 9(1), pp.79-85.<br />

Wats<strong>on</strong>, R.T., Boudreau, M-C and Chen, A.J. (2010) “Informati<strong>on</strong> Systems and Envir<strong>on</strong>mentally Sustainable<br />

Development; Energy Informatics And New Directi<strong>on</strong>s for the IS Community”, MIS Quarterly, 34(1).<br />

Warren McFarlan, F. (1981) “Portfolio Approach to Informati<strong>on</strong> Systems”, Harvard Business Review.<br />

World Commissi<strong>on</strong> <strong>on</strong> Envir<strong>on</strong>ment and Development (1987) Our Comm<strong>on</strong> Future, Oxford University Press,<br />

Great Britain.<br />

301


The art of Shooting the Moving Goal – Explorative Study of<br />

EA Pilot<br />

Nestori Syynimaa<br />

Informatics Research Centre, Henley Business School, University of Reading,<br />

UK<br />

nestori.syynimaa@gmail<br />

Abstract: Enterprise Architecture (EA) has been recognised as an important tool in modern business<br />

management for closing the gap between strategy and its executi<strong>on</strong>. The current literature implies that for EA to<br />

be successful, it should have clearly defined goals. However, the goals of different stakeholders are found to be<br />

different, even c<strong>on</strong>tradictory. In our explorative research, we seek an answer to the questi<strong>on</strong>s: What kind of goals<br />

are set for the EA implementati<strong>on</strong>? How do the goals evolve during the time? Are the goals different am<strong>on</strong>g<br />

stakeholders? How do they affect the success of EA? We analysed an EA pilot c<strong>on</strong>ducted am<strong>on</strong>g eleven Finnish<br />

Higher Educati<strong>on</strong> Instituti<strong>on</strong>s (HEIs) in 2011. The goals of the pilot were gathered from three different stages of<br />

the pilot: before the pilot, during the pilot, and after the pilot, by means of a project plan, interviews during the<br />

pilot and a questi<strong>on</strong>naire after the pilot. The data was analysed using qualitative and quantitative methods. Eight<br />

distinct goals were recognised by the coding: Adopt EA Method, Build Informati<strong>on</strong> Systems, Business<br />

Development, Improve Reporting, Process Improvement, Quality Assurance, Reduce Complexity, and<br />

Understand the Big Picture. The success of the pilot was analysed statistically using the scale 1-5. Results<br />

revealed that goals set before the pilot were very different from those menti<strong>on</strong>ed during the pilot, or after the pilot.<br />

Goals before the pilot were mostly related to expected benefits from the pilot, whereas the most important result<br />

was to adopt the EA method. Results can be explained by possibly different roles of resp<strong>on</strong>dents, which in turn<br />

were most likely caused by poor communicati<strong>on</strong>. Interestingly, goals menti<strong>on</strong>ed by different stakeholders were<br />

not limited to their traditi<strong>on</strong>al areas of resp<strong>on</strong>sibility. For example, in some cases Chief Informati<strong>on</strong> Officers' goals<br />

were Quality Assurance and Process Improvement, whereas managers’ goals were Build Informati<strong>on</strong> Systems<br />

and Adopt EA Method. This could be a result of a good understanding of the meaning of EA, or stakeholders do<br />

not regard EA as their c<strong>on</strong>cern at all. It is also interesting to notice that regardless of the different percepti<strong>on</strong>s of<br />

goals am<strong>on</strong>g stakeholders, all HEIs felt the pilot to be successful. Thus the research does not provide support to<br />

c<strong>on</strong>firm the link between clear goals and success.<br />

Keywords: enterprise architecture, stakeholders, goals, success<br />

1. Introducti<strong>on</strong><br />

The structure of the paper is as follows. Firstly, the problem area is introduced, including the key<br />

c<strong>on</strong>cepts used in the paper. Sec<strong>on</strong>dly, the methodology and data collecti<strong>on</strong> are described. Thirdly,<br />

results of the analysis and discussi<strong>on</strong> are presented, and finally c<strong>on</strong>clusi<strong>on</strong>s are presented.<br />

Enterprise Architecture (EA) has a number of definiti<strong>on</strong>s in the current literature (see for example: CIO<br />

Council 2001; TOGAF 2009; Zachman 1997). We shall adopt the definiti<strong>on</strong> of EA, which is based <strong>on</strong><br />

two comm<strong>on</strong> c<strong>on</strong>cepts shared by the EA definiti<strong>on</strong>s (Syynimaa 2010). Firstly, EA is a formal<br />

descripti<strong>on</strong> of an organisati<strong>on</strong> at a specific time. Usually there are descripti<strong>on</strong>s at least of two different<br />

states of the organisati<strong>on</strong>: current and future. Sec<strong>on</strong>dly, EA is a managed change between these<br />

states. As a descripti<strong>on</strong>, EA is usually described by using a four layer model (Pulkkinen 2006). These<br />

layers are Business Architecture (BA), Informati<strong>on</strong> Architecture (IA), Systems Architecture (SA), and<br />

Technology Architecture (TA).<br />

Lately EA's usability and power in strategy executi<strong>on</strong> has been recognised (Gregor et al. 2007; Ross<br />

et al. 2006). The four layer model of EA uses a top-down approach (Pulkkinen 2006; TOGAF 2009).<br />

Output of a higher level is input for a level below it: BA→ IA→SA→TA. Strategy of an organisati<strong>on</strong> is<br />

described <strong>on</strong> the BA level, so the future state of the organisati<strong>on</strong>'s BA includes possible changes in<br />

the strategy. As such, EA can be used as a tool for closing the gap between the strategy and its<br />

executi<strong>on</strong>.<br />

The current literature implies that for EA implementati<strong>on</strong> to be successful, it should have clearly<br />

defined goals (Iyamu 2009; Martin et al. 2004; Miller 2003). However, the goals of different<br />

stakeholders are found to be different, even c<strong>on</strong>tradictory (van der Raadt et al. 2008). According to<br />

the guide to the Project <strong>Management</strong> Body Of Knowledge (PMBOK), project goals are "the<br />

quantifiable criteria that must be met for the project to be c<strong>on</strong>sidered successful" (Duncan 1996, p.<br />

52). Moreover, unquantifiable goals are of very high risk.<br />

302


Nestori Syynimaa<br />

The rati<strong>on</strong>ale of the research can be summarised as follows. EA has been found to be an important<br />

tool in strategy executi<strong>on</strong>. To implement EA successfully, <strong>on</strong>e should have clear goal(s) set for the<br />

implementati<strong>on</strong>. These goals have been found to be different am<strong>on</strong>g stakeholders, even<br />

c<strong>on</strong>tradictory. General project management literature suggests setting quantifiable goals. Thus in this<br />

exploratory research, we seek answers to the questi<strong>on</strong>s: What kind of goals are set for the EA<br />

implementati<strong>on</strong>? How do the goals evolve during the time? Are those goals different am<strong>on</strong>g<br />

stakeholders? How do they affect the success of EA implementati<strong>on</strong>?<br />

Empirical data was gathered from an EA implementati<strong>on</strong> pilot c<strong>on</strong>ducted am<strong>on</strong>g eleven Finnish<br />

Higher Educati<strong>on</strong> Instituti<strong>on</strong>s (HEIs) during 2011. The goals of the pilot were to start EA work in the<br />

Higher Educati<strong>on</strong> field and to build a basis for a c<strong>on</strong>tinuous EA functi<strong>on</strong> in HEIs. The pilot was<br />

organised in six sub-groups having <strong>on</strong>e or more participants, each focusing <strong>on</strong> a certain topic<br />

(Riihimaa et al. 2011). For instance, <strong>on</strong>e of the groups focused <strong>on</strong> co-operati<strong>on</strong> in teaching and<br />

student movement. In additi<strong>on</strong>, each individual HEI had its own internal focus areas. The structure of<br />

the pilot can be seen in Figure 1.<br />

Figure 1: Pilot structure<br />

2. Methodology<br />

EA implementati<strong>on</strong> is c<strong>on</strong>sidered to be a process, where the initial state of an organisati<strong>on</strong> is changed<br />

from the state before EA t 1 , to the state where EA is implemented t 2 . The process is illustrated in<br />

Figure 2. In this paper, we are performing exploratory research <strong>on</strong> an EA pilot. Methodologically EA<br />

pilot is regarded as an instance of EA implementati<strong>on</strong>, which is executed as a project. Goals are<br />

objectives, targets, etc., set for the pilot. Goals can be measurable (quantifiable) or qualitative<br />

(unquantifiable). Success of any implementati<strong>on</strong> project is found to be difficult to measure<br />

quantifiably, as the c<strong>on</strong>cept of success is too subjective (Cale et al. 1987). We accept this subjective<br />

nature of the c<strong>on</strong>cept of success, and define it as the perceived feeling of success of participating<br />

individuals.<br />

Figure 2: EA implementati<strong>on</strong> (Syynimaa 2012)<br />

Data used in this paper c<strong>on</strong>sisted of a subset of data gathered from three different stages of the pilot<br />

as part of a larger research. Goals before the pilot were gathered from the project plan of the pilot in a<br />

textual form. Goals during the pilot were gathered from interviews, which were c<strong>on</strong>ducted as ph<strong>on</strong>e<br />

interviews. Interviews were recorded and transcribed. Three different roles were interviewed in each<br />

HEI: Chief Informati<strong>on</strong> Officers (CIOs), managers (president, rector, etc.), and Quality Assurance (QA)<br />

staff. A semi-structured interview technique was used, where interviewees were given a certain theme<br />

to answer. In the case of goals, the questi<strong>on</strong>/theme asked was (translated from Finnish): “With regard<br />

to the pilot, what are your or your instituti<strong>on</strong>’s goals for the pilot?” Goals after the pilot were gathered<br />

from a questi<strong>on</strong>naire sent to the pilot’s project and steering group members two m<strong>on</strong>ths after the pilot.<br />

In the questi<strong>on</strong>naire the goals of the pilot were asked as an open ended questi<strong>on</strong> from four different<br />

perspectives: pers<strong>on</strong>al goals, instituti<strong>on</strong>’s goals, group’s goals, and pilot’s goals. Also Most important<br />

results and the success of the pilot were gathered from the very same questi<strong>on</strong>naire.<br />

303


Nestori Syynimaa<br />

All textual data was coded using the open-coding technique used in Grounded Theory (Glaser et al.<br />

1967). As the purpose was simply to categorise similar goals under the same code, no axial or<br />

selective coding was used. During the coding, a new category was added if the goal did not fit to any<br />

existing goal. Thus, some codes may be overlapping, or even from different category layers. The<br />

perceived success of the pilot was arrived at by using a Likert scale (1-5) questi<strong>on</strong>.<br />

3. Results and discussi<strong>on</strong><br />

The unit of analysis in this research is the sub-group. The sub-group level was used for two reas<strong>on</strong>s.<br />

First of all, there is data available from different states <strong>on</strong>ly at the sub-group level. Sec<strong>on</strong>dly, usage of<br />

the sub-group level helps us to hide identities of HEIs, as some goals could be c<strong>on</strong>nected to a certain<br />

HEI. One HEI formed a separate sub-group but was also a member of another sub-group having<br />

multiple members. This HEI has been analysed as a part of the latter sub-group <strong>on</strong>ly, thus the total<br />

number of analysed sub-groups is five.<br />

During the coding eight distinct goals were found: Adopt EA Method, Build Informati<strong>on</strong> Systems,<br />

Business Development, Improve Reporting, Process Improvement, Quality Assurance, Reduce<br />

Complexity, and Understand the Big Picture. Adopt EA Method means goals related to adopting,<br />

learning, and introducing the EA method. In these cases, the goal of the EA pilot is to adopt the EA<br />

method per se, without any “greater” goal. Build Informati<strong>on</strong> Systems refers to goals for building an<br />

informati<strong>on</strong> system. Business Development refers to goals for business development, for instance by<br />

a comparis<strong>on</strong> to other HEIs, or sustaining competitiveness by merging some functi<strong>on</strong>s with another<br />

HEI. Improve Reporting refers to goals for improvement of reporting in terms of automati<strong>on</strong>, quality<br />

and ease of access. Process Improvement refers to goals for improvement of HEI's processes,<br />

whether business or informati<strong>on</strong> and communicati<strong>on</strong> technology (ICT) processes. Quality Assurance<br />

refers to goals related to QA functi<strong>on</strong> and its activities. Reduce Complexity refers to goals for reducing<br />

complexity of either processes or informati<strong>on</strong> systems. Understand the Big Picture refers to goals for<br />

understanding the big picture of the HEI as a whole, including informati<strong>on</strong> systems. Examples of the<br />

codes can be seen in Table 1.<br />

Table 1: Codes and examples of goals (translated from Finnish)<br />

Code Goal<br />

Adopt EA Method "Familiarising ourselves with EA framework"<br />

Build Informati<strong>on</strong> Systems "To build a shared data warehouse for all Higher Educati<strong>on</strong> Instituti<strong>on</strong>s"<br />

Business Development "Benchmarking to other HEIs"<br />

Improve Reporting "To ease reporting"<br />

Process Improvement "To support process based development of informati<strong>on</strong> systems"<br />

Quality Assurance "To prepare for QA audit"<br />

Reduce Complexity "Reducing overlapping work"<br />

Understand the Big Picture "To better understand c<strong>on</strong>sequences of our decisi<strong>on</strong>s"<br />

Results of the analysis can be seen in Table 2, where each row represents a sub-group. The three<br />

next columns, Before, During, and After, refer to goals set for the pilot. The fourth column, Results,<br />

refers to the most important results of the pilot. The last column refers to the perceived success of the<br />

pilot <strong>on</strong> a scale of 1-5. The values of the success column are medians of resp<strong>on</strong>dents’ answers of the<br />

particular sub-group. In the During column, resp<strong>on</strong>dents’ role(s) are also given. These roles are:<br />

C=Chief Informati<strong>on</strong> Officer, M=<strong>Management</strong> (principal, rector, president), Q=Quality Assurance staff.<br />

Table 2: Results<br />

Before During After Results Suc.<br />

Adopt EA method<br />

Quality assurance<br />

Business<br />

development<br />

Adopt EA method (C<br />

Build informati<strong>on</strong> systems (MQ<br />

Business development (M<br />

Process improvement (M<br />

Reduce complexity (MQ<br />

Adopt EA method (M<br />

Improve reporting (M<br />

Process improvement (C<br />

Quality Assurance (M<br />

Understand the big picture (C<br />

304<br />

Adopt EA method<br />

Business<br />

development<br />

Process<br />

improvement<br />

Adopt EA method<br />

Business<br />

development<br />

Process<br />

improvement<br />

Understand the big<br />

picture<br />

Adopt EA<br />

method<br />

4<br />

Adopt EA<br />

method<br />

Understand<br />

the big picture 4


Nestori Syynimaa<br />

Before During After Results Suc.<br />

Adopt EA method Business development (CM<br />

Reduce complexity (CM<br />

Improve reporting (Q<br />

Quality assurance (Q<br />

Build informati<strong>on</strong> systems (M<br />

Process improvement (M<br />

Understand the big picture (M<br />

Process improvement<br />

Understand the big<br />

picture<br />

Build informati<strong>on</strong><br />

systems<br />

Quality assurance<br />

Adopt EA method (M<br />

Build informati<strong>on</strong> systems (C<br />

Business development (M<br />

Improve reporting (M<br />

Process improvement (CQ<br />

Understand the big picture (CM<br />

Quality assurance (C<br />

Adopt EA method (CM<br />

Business development (M<br />

Process improvement (CQ<br />

Understand the big picture (Q<br />

Adopt EA method<br />

Process<br />

improvement<br />

Improve reporting<br />

Adopt EA<br />

method<br />

Process<br />

improvement 3<br />

Adopt EA method Adopt EA<br />

method<br />

Adopt EA method<br />

Business<br />

development<br />

Process<br />

improvement<br />

Quality assurance<br />

Understand the big<br />

picture<br />

Adopt EA<br />

method<br />

Business<br />

development<br />

Results summarised in Table 2 lead us to the following findings. Before the pilot, goals were mostly<br />

related to the expected outcomes of the EA pilot. This was the case in four out of five sub-groups.<br />

Adopting the EA method was menti<strong>on</strong>ed in the goals of <strong>on</strong>ly two sub-groups. It should be noted that<br />

the project plan, which was a source for before-the-pilot data, was composed mainly by CIOs.<br />

Moreover, its purpose was to "sell" the project to HEIs' management.<br />

During the pilot, there were a lot more goals menti<strong>on</strong>ed than before the pilot. There were a number of<br />

resp<strong>on</strong>dents, mainly managers and QA staff, who were not members of project groups. Thus their<br />

view of the pilot’s goals was based solely <strong>on</strong> internal communicati<strong>on</strong> and publicly available material.<br />

Variance of the answers can be explained by this to some degree. However, it does not explain why<br />

goals menti<strong>on</strong>ed by CIOs are different from those before the pilot. It is also interesting to note that in<br />

some cases goals are not related to resp<strong>on</strong>dents’ own duties. For example, in some cases CIOs’<br />

goals were Quality Assurance and Process Improvement, whereas managers’ goals were Build<br />

Informati<strong>on</strong> Systems and Adopt EA Method. This could be a result of a good understanding of the<br />

meaning of EA, or that stakeholders do not regard EA as their c<strong>on</strong>cern at all.<br />

Goals after the pilot were gathered from a questi<strong>on</strong>naire sent to the pilot's steering and project<br />

groups. Thus all resp<strong>on</strong>dents should have been aware of the goals before the pilot. Still, most of the<br />

goals menti<strong>on</strong>ed were related to EA adopti<strong>on</strong>. This was also the case when asking the most important<br />

results of the pilot. All sub-groups menti<strong>on</strong>ed the adopti<strong>on</strong> of the EA method as <strong>on</strong>e of the most<br />

important results of the pilot. Two of the sub-groups menti<strong>on</strong>ed <strong>on</strong>ly the EA method, while the rest of<br />

the sub-groups also menti<strong>on</strong>ed another goal. Goals after the pilot and most important results were<br />

gathered <strong>on</strong> the same questi<strong>on</strong>naire, which explains their similarities as all of the results were also<br />

menti<strong>on</strong>ed as goals.<br />

The most interesting finding is that there is no single sub-group which menti<strong>on</strong>ed even a single goal in<br />

all the stages and as the most important result. Moreover, in <strong>on</strong>ly two cases was <strong>on</strong>e of the beforethe-pilot<br />

goals menti<strong>on</strong>ed. This could be interpreted as a failure, but not a single sub-group perceived<br />

the pilot as being a failure. Findings of the research can be summarised as follows. Goals set to EA<br />

implementati<strong>on</strong> evolve during the implementati<strong>on</strong> project. There is also a notable variance of the<br />

goals am<strong>on</strong>g different stakeholders. Development of the goals and their variance am<strong>on</strong>g stakeholders<br />

does not seem to affect the perceived success of the implementati<strong>on</strong>. It is fair to put the questi<strong>on</strong> why<br />

the EA pilot was perceived as being a success, when all the participants felt the adopti<strong>on</strong> of the EA<br />

method was the most important result. Half of the participants also had some business results, but not<br />

even <strong>on</strong>e of those were original goals menti<strong>on</strong>ed before the pilot. Does it mean that Enterprise<br />

Architecture does not provide business results at all? Or does it mean that business outcomes are felt<br />

to be so natural a result of the EA implementati<strong>on</strong>, that <strong>on</strong>ly that was seen as important?<br />

305<br />

4<br />

3


4. C<strong>on</strong>clusi<strong>on</strong>s<br />

Nestori Syynimaa<br />

Previous research <strong>on</strong> EA implementati<strong>on</strong> has shown that clear goals set for the implementati<strong>on</strong> are<br />

<strong>on</strong>e of the key success factors (Iyamu 2009; Martin et al. 2004; Miller 2003). Also communicati<strong>on</strong><br />

during the implementati<strong>on</strong> has been found to be a very important factor (Gregor et al. 2007; Iyamu<br />

2009; Kaisler et al. 2005; Richards<strong>on</strong> et al. 1990; Shupe et al. 2006; van der Raadt et al. 2009). The<br />

research findings show that regardless of the different percepti<strong>on</strong>s of goals am<strong>on</strong>g stakeholders, all<br />

sub-groups felt the pilot was successful. Thus the research does not provide support to c<strong>on</strong>firm the<br />

link between clear goals and success. What it clearly indicates though is that communicati<strong>on</strong> plays a<br />

key role in the implementati<strong>on</strong>, which can be seen in the variance of goals menti<strong>on</strong>ed during the pilot.<br />

The author acknowledges the limitati<strong>on</strong>s of the research, especially in generalising the findings. The<br />

exploratory nature of the research limits the applicability of the findings str<strong>on</strong>gly to the c<strong>on</strong>text where it<br />

was c<strong>on</strong>ducted. However, the power of exploratory research is in its ability to raise more questi<strong>on</strong>s<br />

than it can answer. Research has therefore more scientific than practical implicati<strong>on</strong>s. This research<br />

for instance introduces some observati<strong>on</strong>s that are likely to be present also in a wider c<strong>on</strong>text, and<br />

thus can provide an interesting area for further research. For instance, the effect of clear (or unclear)<br />

goals to the success of EA implementati<strong>on</strong> requires more systematic research.<br />

References<br />

Cale, E. G., and Curley, K. F. "Measuring Implementati<strong>on</strong> Outcome: Bey<strong>on</strong>d Success and Failure," Informati<strong>on</strong> &<br />

<strong>Management</strong> (13:5) 1987, pp 245-253.<br />

CIO Council "A Practical Guide to Federal Enterprise Architecture," Available at<br />

http://www.cio.gov/documents/bpeaguide.pdf) 2001.<br />

Duncan, W. R. A Guide to the Project <strong>Management</strong> Body of Knowledge PMI Publishing Divisi<strong>on</strong>, Sylva, North<br />

Carolina, USA, 1996, p. 182.<br />

Glaser, B. G., and Strauss, A. L. The Discovery Of Grounded Theory, Strategies for Qualitative Research. Aldine<br />

Publishers, Chicago, 1967.<br />

Gregor, S., Hart, D., and Martin, N. "Enterprise architectures: enablers of business strategy and IS/IT alignment<br />

in government," Informati<strong>on</strong> Technology & People (20:2) 2007, pp 96-120.<br />

Iyamu, T. "Strategic Approach for the Implementati<strong>on</strong> of Enterprise Architecture: A Case Study of Two<br />

Organizati<strong>on</strong>s in South Africa," ICISO. 11th Internati<strong>on</strong>al <str<strong>on</strong>g>C<strong>on</strong>ference</str<strong>on</strong>g> <strong>on</strong> Informatics and Semiotics in<br />

Organisati<strong>on</strong>s, Beijing University of Technology, Beijing, China, 2009, pp. 375-381.<br />

Kaisler, H., Armour, F., and Valivullah, M. "Enterprise Architecting: Critical Problems," HICSS-38. Proceedings of<br />

the 38th Annual Hawaii Internati<strong>on</strong>al <str<strong>on</strong>g>C<strong>on</strong>ference</str<strong>on</strong>g> <strong>on</strong> System Sciences, Waikoloa, Hawaii, USA, 2005.<br />

Martin, N., Gregor, S., and Hart, D. "Using a comm<strong>on</strong> architecture in Australian e-Government: The Case of<br />

Smart Service Queensland," ICEC'04. Proceedings of the <str<strong>on</strong>g>6th</str<strong>on</strong>g> Internati<strong>on</strong>al <str<strong>on</strong>g>C<strong>on</strong>ference</str<strong>on</strong>g> <strong>on</strong> Electr<strong>on</strong>ic<br />

Commerce, ACM, Delft, The Netherlands, 2004, pp. 516-525.<br />

Miller, P. C. "Enterprise architecture implementati<strong>on</strong> in a state government," University of Phoenix, Phoenix,<br />

Ariz<strong>on</strong>e, United States, 2003, p. 132.<br />

Pulkkinen, M. "Systemic <strong>Management</strong> of Architectural Decisi<strong>on</strong>s in Enterprise Architecture Planning. Four<br />

Dimensi<strong>on</strong>s and Three Abstracti<strong>on</strong> Levels. System Sciences, 2006. HICSS’06," 2006.<br />

Richards<strong>on</strong>, G. L., Jacks<strong>on</strong>, B. M., and Dicks<strong>on</strong>, G. W. "A Principles-Based Enterprise Architecture: Less<strong>on</strong>s<br />

from Texaco and Star Enterprise," MIS Quarterly (14:4) 1990, pp 385-403.<br />

Riihimaa, J., and Syynimaa, N. "Enterprise Architecture Framework Adopti<strong>on</strong> By Finnish Applied Universities'<br />

Network," in: EUNIS 2011, Dublin, Ireland, 2011.<br />

Ross, J. W., Weill, P., and Roberts<strong>on</strong>, D. C. Enterprise architecture as strategy: Creating a foundati<strong>on</strong> for<br />

business executi<strong>on</strong> Harvard Business School Press, Bost<strong>on</strong>, Massachusetts, USA, 2006.<br />

Shupe, C., and Behling, R. "Developing and Implementing a Strategy for Technology Deployment," Informati<strong>on</strong><br />

<strong>Management</strong> Journal (40:4) 2006, pp 52-57.<br />

Syynimaa, N. "Tax<strong>on</strong>omy of purpose of Enterprise Architecture " in: 12th Internati<strong>on</strong>al <str<strong>on</strong>g>C<strong>on</strong>ference</str<strong>on</strong>g> <strong>on</strong> Informatics<br />

and Semiotics in Organisati<strong>on</strong>s, ICISO 2010, Reading, UK, 2010.<br />

Syynimaa, N. "Measuring Enteprise Architecture Success: Tentative Model for Measuring EA Implementati<strong>on</strong><br />

Success " in: IRIS 2012, Sigtuna, Sweden, 2012.<br />

TOGAF TOGAF Versi<strong>on</strong> 9 Van Haren Publishing, 2009.<br />

van der Raadt, B., Schouten, S., and van Vliet, H. "Stakeholder percepti<strong>on</strong> of enterprise architecture," ECSA<br />

2008, Sec<strong>on</strong>d <str<strong>on</strong>g>European</str<strong>on</strong>g> <str<strong>on</strong>g>C<strong>on</strong>ference</str<strong>on</strong>g> <strong>on</strong> Software Architecture, Springer, Paphos, Cyprus, 2008, pp. 19-34.<br />

van der Raadt, B., and van Vliet, H. "Assessing the Efficiency of the Enterprise Architecture Functi<strong>on</strong>," in:<br />

Advances in Enterprise Engineering II. First NAF Academy Working <str<strong>on</strong>g>C<strong>on</strong>ference</str<strong>on</strong>g> <strong>on</strong> Practice-Driven<br />

Research <strong>on</strong> Enterprise Transformati<strong>on</strong>, PRET 2009, held at CAiSE 2009, Amsterdam, The Netherlands,<br />

June 11, 2009. Proceedings, E. Proper, F. Harmsen and J. L.G.Dietz (eds.), Springer Berlin Heidelberg,<br />

Amsterdam, The Netherlands, 2009, pp. 63-83.<br />

Zachman, J. A. "Enterprise architecture: The issue of the century," Database Programming and Design (10:3)<br />

1997, pp 44-53.<br />

306


Informati<strong>on</strong> Interacti<strong>on</strong> in Terms of eCommerce<br />

Kamila Tislerova<br />

Technical University of Liberec, Liberec, Czech Republic<br />

Kamila.tislerova@tul.cz<br />

Abstract: This paper deals with some aspects of the informati<strong>on</strong> data resulting from the relati<strong>on</strong>ship between an<br />

Internet enterprise (e-shop) and the customer. Analyzing and evaluating informati<strong>on</strong> which is provided and<br />

required enables <strong>on</strong>e to derive important recommendati<strong>on</strong>s for creating and managing a user-friendly and thus<br />

probably successful Internet business. There are many recommendati<strong>on</strong>s in Customer Relati<strong>on</strong>ship <strong>Management</strong><br />

emphasizing the importance of maintaining post-purchase c<strong>on</strong>tact with customers; some businesses<br />

automatically accompany the purchase by a questi<strong>on</strong>naire c<strong>on</strong>cerning customer satisfacti<strong>on</strong>. Are they sure<br />

customers welcome such a c<strong>on</strong>tact? Could some informati<strong>on</strong> interacti<strong>on</strong> be annoying for the customers? There<br />

are three research questi<strong>on</strong>s to be answered: firstly how the after-purchase c<strong>on</strong>tact is perceived by customers.<br />

The sec<strong>on</strong>d questi<strong>on</strong> relates to who the customers providing informati<strong>on</strong> to the business actually are; their<br />

descripti<strong>on</strong> c<strong>on</strong>cerns both their demographic descripti<strong>on</strong> and their purchase behaviour. Thirdly, how significant in<br />

reality the informati<strong>on</strong> provided by eCommerce businesses for the customers actually is? This research was<br />

undertaken and involved more than 500 resp<strong>on</strong>dents. The questi<strong>on</strong>naire c<strong>on</strong>sists of both qualitative and<br />

quantitative issues so that a large variety of results can be derived from it. The resp<strong>on</strong>dents are the <strong>on</strong>ly people<br />

having experience with using eCommerce enterprises for their shopping. In the survey, they provide informati<strong>on</strong><br />

<strong>on</strong> their preferences, purchase behaviour, customs, perceived risk-taking and willingness to share informati<strong>on</strong><br />

with the enterprises. In additi<strong>on</strong> to a comm<strong>on</strong> dependence search, a factor analysis was also carried out in order<br />

to better understand and formulate the customer approach towards informati<strong>on</strong> interacti<strong>on</strong>. Because this paper<br />

also provides results pertaining to the identificati<strong>on</strong> of some attributes of user-friendly eCommerce enterprises<br />

(examined from the customers’ point of view), its results should c<strong>on</strong>tribute to the general discussi<strong>on</strong> c<strong>on</strong>cerning<br />

the usage of informati<strong>on</strong> in business-customer relati<strong>on</strong>s. <strong>Academic</strong>s and researchers should be placed at an<br />

advantage by virtue of some of the c<strong>on</strong>clusi<strong>on</strong>s arising out of a large survey am<strong>on</strong>g customers who are<br />

accustomed to executing purchases via eCommerce businesses. The specific findings of this paper should be<br />

helpful for marketing practiti<strong>on</strong>ers in their marketing and communicati<strong>on</strong> mix creati<strong>on</strong>. It should help to establish a<br />

system of customer care in terms of informati<strong>on</strong> interchange which should be highly appreciated by customers<br />

as well as prove highly beneficial for enterprises.<br />

Keywords: informati<strong>on</strong> interacti<strong>on</strong>; eCommerce; customer relati<strong>on</strong>ship management; informati<strong>on</strong> sharing<br />

1. Introducti<strong>on</strong><br />

Informati<strong>on</strong> interacti<strong>on</strong> between business and customer is a crucial matter for effective businesses<br />

(Kotler, 2004). By sharing relevant informati<strong>on</strong>, the business is able to facilitate the c<strong>on</strong>sumer's<br />

purchase process and thus to realize a so called co-creati<strong>on</strong> of value. Value is created when a<br />

c<strong>on</strong>sumer is offered useful informati<strong>on</strong> and gains understanding, reassurance and/or hed<strong>on</strong>ic<br />

fulfilment in the process (Grant et al., 2007). Value creati<strong>on</strong> relies <strong>on</strong> an analysis of <strong>on</strong>line c<strong>on</strong>sumer<br />

behaviour to determine which informati<strong>on</strong> sources and formats are most likely to meet their needs at a<br />

given point in time (Teo at al., 2004). The approach follows the view of Payne et al. (2008): that a<br />

customer becomes a co-creator of value through the development of customer–supplier relati<strong>on</strong>ships<br />

based <strong>on</strong> informati<strong>on</strong> interacti<strong>on</strong> and dialogue. C<strong>on</strong>tributi<strong>on</strong>s given by customers to businesses can<br />

be measured and quantified by several existing methods (Tislerova, 2011), so that customers´<br />

cooperati<strong>on</strong> should be c<strong>on</strong>sidered as a profitable item. Thus, communicati<strong>on</strong> and informati<strong>on</strong><br />

interacti<strong>on</strong> become crucial issue for business development.<br />

2. Informati<strong>on</strong> interacti<strong>on</strong> in society<br />

Informati<strong>on</strong> and communicati<strong>on</strong> technologies (ICT) have changed our lives dramatically. What<br />

seemed impossible has become a crucial part of today’s reality. Proper use of ICT can enhance<br />

competitiveness not <strong>on</strong>ly of whole countries and regi<strong>on</strong>s, but also of companies and individuals.<br />

Am<strong>on</strong>g others, there are two important frameworks depicting role of ICT in society. The first <strong>on</strong>e was<br />

defined by OECD and is called “Informati<strong>on</strong> Society Statistics C<strong>on</strong>ceptual Model” (OECD, 2011). It<br />

tries to describe the complexity of the informati<strong>on</strong> society and includes all major entities and relati<strong>on</strong>s<br />

somehow related to ICT. Electr<strong>on</strong>ic trade is included in a group labeled “ICT Demand (Users and<br />

Use)”. This framework is also related to “Informati<strong>on</strong> Society Impacts Measurement Model” (OECD,<br />

2008). It indicates areas of interest that are divided into two groups—easier (ec<strong>on</strong>omic, positive,<br />

short-term, direct, and narrow impacts) and harder (social, negative, macro, l<strong>on</strong>g-term, and<br />

unintended impacts) to measure. The United Nati<strong>on</strong>s formulated an “ICT Impact Relati<strong>on</strong>ships Model”<br />

307


Kamila Tislerova<br />

that comprises relati<strong>on</strong>s bel<strong>on</strong>ging to the three main fields: ec<strong>on</strong>omy, society and envir<strong>on</strong>ment<br />

(UNCTAD, 2011).<br />

2.1 Informati<strong>on</strong> interacti<strong>on</strong> in terms of eCommerce<br />

Exploring informati<strong>on</strong> interacti<strong>on</strong> in terms of eCommerce differs greatly in comparis<strong>on</strong> with traditi<strong>on</strong>al<br />

informati<strong>on</strong> interacti<strong>on</strong>. E-business usually implies a rethinking of business models, of the network,<br />

and system infrastructure. Therefore, <strong>on</strong>ly businesses with access to significant e-business<br />

competency in informati<strong>on</strong> treatment can expect to succeed in their efforts (Daniel, Wils<strong>on</strong>, 2003).<br />

Research <strong>on</strong> eCommerce and business-customer relati<strong>on</strong>ship indicates that both fields have<br />

significant impacts <strong>on</strong> ec<strong>on</strong>omic growth and wealth creati<strong>on</strong> (Acs et al., 2004).<br />

ECommerce is growing at an incredible pace. The accessibility of the Internet makes electr<strong>on</strong>ic<br />

commerce a realistic possibility for ec<strong>on</strong>omic growth and business development. As the amount of<br />

business transacted over the web increases, the value of goods, services, and informati<strong>on</strong> exchange<br />

over the Internet seems to double or triple each year around the globe (Kathuria, Joshi 2007). The<br />

Internet has truly transformed the way c<strong>on</strong>sumers shop in a multitude of categories (e.g. travel,<br />

books, videos) and the way most retailers do business with their suppliers as well as their customers.<br />

More and more successful retailers are leveraging the power of the Internet by improving<br />

effectiveness in informati<strong>on</strong> interacti<strong>on</strong> between businesses and their customers, e.g. providing <strong>on</strong>line<br />

customizati<strong>on</strong>, publishing <strong>on</strong>line flyers and promoti<strong>on</strong>s. On the other side, c<strong>on</strong>sumers use the Internet<br />

to shop more effectively (Puccinelli et al. 2009).<br />

2.2 Importance of customer feedback<br />

It is vital for any business to have access to customer feedback in order to determine the overall<br />

experience of those who are using their particular goods or services. Customer feedback enables a<br />

company to learn what customers thought of their experience dealing with the company (Mo<strong>on</strong> at al.,<br />

2004). Equipped with this valuable informati<strong>on</strong>, companies can either change their policies to better<br />

suit the customer experience, or build up<strong>on</strong> the strengths that are outlined in the feedback they<br />

receive. The feedback can be collected by means of a customer survey, which might be expensive, or<br />

by the most widely accepted and most effective form of collecting feedback – regular e-feedback<br />

accompanying each purchase. However, this c<strong>on</strong>tact might be also annoying or negatively perceived<br />

by customers.<br />

Also the informati<strong>on</strong> flow from company to customer has often been excessive. Marketers try to<br />

deliver as much informati<strong>on</strong> as possible together with the product or services. Unfortunately, it is not<br />

the most efficient way of communicating with customers. Some informati<strong>on</strong> is really valuable for<br />

customers and required by customers but there is a huge volume of informati<strong>on</strong> explosi<strong>on</strong> the<br />

customer cannot absorb. Thus, a hierarchical approach is highly desirable according to the type of<br />

customer; characteristics of the goods or services bought and in additi<strong>on</strong> to this, many other aspects<br />

should be taken into account.<br />

In terms of eCommerce there is an easy way to acquire the customer feedback that businesses<br />

require without overloading the customers or bombarding them with frequent feedback<br />

questi<strong>on</strong>naires. Customers also have to c<strong>on</strong>fr<strong>on</strong>t the informati<strong>on</strong> explosi<strong>on</strong> provided by businesses; a<br />

proper balance should be established in order to gain an effective way of cooperati<strong>on</strong> which leads to<br />

the benefit of both sides.<br />

2.3 ICT in transiti<strong>on</strong> ec<strong>on</strong>omies<br />

Sustained ec<strong>on</strong>omic development requires a well-developed infrastructure and a substantial number<br />

of high-value-added industries. Thus in developing ec<strong>on</strong>omies, ICT is often regarded as an enabler<br />

and catalyst for successfully shifting away from ec<strong>on</strong>omic dependency <strong>on</strong> low-value-added industry<br />

sectors, such as agriculture and raw materials extracti<strong>on</strong>. ICT as a communicati<strong>on</strong> and collaborati<strong>on</strong>enabling<br />

tool may be profitable in developed countries provides some additi<strong>on</strong>al advantages, aside<br />

from saving cost. As much of the populati<strong>on</strong> in rural areas of developing countries is unskilled in the<br />

use of modern technology, sharing such technology provides not <strong>on</strong>ly access to the technology itself,<br />

but also user support for the technology. The strategic objectives for ICT investments in developing<br />

ec<strong>on</strong>omies are also often different from objectives in developed countries. In emerging ec<strong>on</strong>omies<br />

ICT is used to support the development of new products and services for a rapidly growing customer<br />

308


Kamila Tislerova<br />

base. In c<strong>on</strong>trast, in developed, mature ec<strong>on</strong>omies where the ec<strong>on</strong>omic growth is rather modest, ICT<br />

is primarily used for improvement of existing products and services and to manage the existing<br />

customers more efficiently. Overall, though the use of ICT in developed and in transiti<strong>on</strong> countries<br />

differs substantially, ICT plays a critical role in business growth in developing ec<strong>on</strong>omies (Roztocki, N.<br />

Weistroffer, 2009).<br />

2.4 ECommerce in the Czech Republic<br />

Sales in 2011 are based <strong>on</strong> estimates. There was a steady rise of electr<strong>on</strong>ic trade importance with<br />

<strong>on</strong>ly slight reducti<strong>on</strong> in sales growth rate in the year 2011. An approximate share of B2C eCommerce<br />

<strong>on</strong> retail trade in 2010 was around 4%. The share of individuals purchasing <strong>on</strong>line <strong>on</strong> populati<strong>on</strong> is<br />

higher in EU-27 than in the Czech Republic: 43% compared to <strong>on</strong>ly 30% (Eurostat, 2011), but<br />

shopping <strong>on</strong>line becomes more and more popular, and more than 95% of Czech internet users buy<br />

Christmas gifts <strong>on</strong>line. Overall, the total number of e-shops operating <strong>on</strong> the Czech market is<br />

approximately between 10,000—23,000 subjects, but <strong>on</strong>ly 500—1,000 of them operate as “full-time”<br />

e-shops, i.e. companies focusing <strong>on</strong>ly <strong>on</strong> eCommerce. New <strong>on</strong>es are being created every day, but an<br />

apparent pressure to c<strong>on</strong>solidate is noticeable. C<strong>on</strong>sumers tend to change their behaviour (Dedkova,<br />

2010) and especially retail industry is developed and internati<strong>on</strong>alized very rapidly (Simova, 2010).<br />

Furthermore, every enterprise should search for more effective way of doing business, so that<br />

eCommerce is becoming a natural part of business activity of small and medium sized enterprises<br />

(Rydvalova, Marsikova 2011).<br />

3. Research questi<strong>on</strong>s and methodology<br />

In order to c<strong>on</strong>tribute to a more efficient method of informati<strong>on</strong> data exchange, three research<br />

questi<strong>on</strong>s were laid out: firstly, how the after-purchase c<strong>on</strong>tact is perceived by customers. The sec<strong>on</strong>d<br />

questi<strong>on</strong> relates to who the customers providing informati<strong>on</strong> to the business actually are. Thirdly, how<br />

significant the informati<strong>on</strong> provided by eCommerce businesses for the customers in reality is.<br />

More then 500 Internet shoppers were examined in this survey which was carried out in the Czech<br />

Republic. Resp<strong>on</strong>dents provided informati<strong>on</strong> <strong>on</strong> their preferences, purchase behaviour, habits,<br />

perceived risk-taking and willingness to share informati<strong>on</strong> with the enterprises. In total there were 58<br />

questi<strong>on</strong>s both qualitative and quantitative. The structure of the resp<strong>on</strong>dents was modified according<br />

to the estimated structure of the populati<strong>on</strong> realizing their shopping via the Internet; adjustments and<br />

correcti<strong>on</strong>s were made according to age structure (younger), level of educati<strong>on</strong> (higher) and also<br />

income structure (higher). The other indicators were kept for applicati<strong>on</strong> to the regular populati<strong>on</strong>.<br />

After two focus groups sessi<strong>on</strong>s for precisely determining the range of potential answers, the<br />

questi<strong>on</strong>naire was drawn up, placed in web pages and distributed in the form of a link to these web<br />

pages. Missing answers were also included in order to raise the level of trustworthiness of the data to<br />

the highest level possible.<br />

The data was processed by statistical methods such as descriptive statistics, correlati<strong>on</strong>s, crosstabs<br />

and factor analysis. Also tests of normality and other relevant tests were applied in accordance with<br />

data processing principles (Zambochova, 2008).<br />

4. Findings c<strong>on</strong>cerning after-purchase c<strong>on</strong>tact percepti<strong>on</strong><br />

Participants of the survey expressed their percepti<strong>on</strong> of businesses´ request for providing feedback,<br />

usually in the form of an e-questi<strong>on</strong>naire.<br />

The questi<strong>on</strong> was formulated in the form of a scale – the strength of c<strong>on</strong>sent with the statement:<br />

“After-purchase c<strong>on</strong>tact is not annoying for me”. The distributi<strong>on</strong> of answers is shown in Table 1:<br />

Regarding the gender distributi<strong>on</strong> of perceived annoyance resulting from the request for feedback: In<br />

total the situati<strong>on</strong> seems to be well balanced (Males - 46.6% and Females 53.4%). However, a<br />

substantial difference occurs in the degree of tolerance as a result of this after-purchase c<strong>on</strong>tact.<br />

Whereas 69.4% of males str<strong>on</strong>gly agree (c<strong>on</strong>tact is not annoying) and females are in the minority, a<br />

neutral percepti<strong>on</strong> predominates am<strong>on</strong>g females (61.6%).<br />

309


Kamila Tislerova<br />

Figure 1: Distributi<strong>on</strong> of resp<strong>on</strong>dents - perceived as annoyance resulting from after-purchase<br />

questi<strong>on</strong>ing<br />

Table 1: Structure of resp<strong>on</strong>dents according to their request for feedback percepti<strong>on</strong><br />

Valid<br />

Frequency Percent<br />

Cumulative<br />

Percent<br />

str<strong>on</strong>gly agree 78 15.5 15.5<br />

likely to agree 146 29.0 44.5<br />

depends <strong>on</strong> the situati<strong>on</strong> 172 34.2 78.7<br />

Likely to disagree 71 14.1 92.8<br />

str<strong>on</strong>gly disagree 36 7.2 100.0<br />

Total 503 100.0<br />

No significant differences were found in the examinati<strong>on</strong> of the dependency of the percepti<strong>on</strong> of afterpurchase<br />

c<strong>on</strong>tact am<strong>on</strong>g the differing educati<strong>on</strong>al levels. The current working positi<strong>on</strong> <strong>on</strong> the other<br />

hand influences the percepti<strong>on</strong> when the most “tolerant” group is made up of students of higher<br />

schools and ordinary employees (28.3%, resp. 29%). The less resp<strong>on</strong>sive category is formed from<br />

self-employed resp<strong>on</strong>dents (5.7%).<br />

Is the positive (or neutral) percepti<strong>on</strong> with regard to receiving requests for feedback dependent <strong>on</strong> the<br />

period of time the resp<strong>on</strong>dent requires to realize his/her purchases via Internet? Answers vary from 1<br />

to 11 years, the mean is 4.69 years (although showing a relatively large Standard deviati<strong>on</strong> of 2.468).<br />

In all, half of the resp<strong>on</strong>dents (49.5) accepting feedback, had been realizing their Internet purchase<br />

maximally for four years.<br />

The other questi<strong>on</strong> was formulated: When you receive request for feedback (e-questi<strong>on</strong>naire), what is<br />

your reacti<strong>on</strong>? There were five opti<strong>on</strong>s for the answer: always to provide a resp<strong>on</strong>se, usually to<br />

provide a resp<strong>on</strong>se, it depends <strong>on</strong> the situati<strong>on</strong>, usually do not provide a resp<strong>on</strong>se and never provide<br />

a resp<strong>on</strong>se. The distributi<strong>on</strong> of answers is very similar to the histogram of answers which examines<br />

the percepti<strong>on</strong> of the request for feedback (after-purchase c<strong>on</strong>tact).<br />

5. Findings relating to some characteristics of resp<strong>on</strong>dents providing<br />

informati<strong>on</strong> to businesses<br />

For the purpose of this survey, two main categories of informati<strong>on</strong> transfer were created: So-called<br />

“alerts providers” and “discussi<strong>on</strong> c<strong>on</strong>tributors”. Based <strong>on</strong> the questi<strong>on</strong> “When noticing a problem or<br />

cause for some discomfort to the customer, do you inform (alert) the business?” the first group was<br />

310


Kamila Tislerova<br />

identified. A three-rate scale was used in both questi<strong>on</strong>s (often, occasi<strong>on</strong>ally, never). The sec<strong>on</strong>d<br />

questi<strong>on</strong> asks: “Do you c<strong>on</strong>tribute to discussi<strong>on</strong>s, share your knowledge and experience?”<br />

Figure 2: Share (in %) of resp<strong>on</strong>dents reacting in the form of “alerts” and “discussi<strong>on</strong>”<br />

Less then half (46.3%) of the resp<strong>on</strong>dents often or occasi<strong>on</strong>ally share their experience, provide<br />

advice and c<strong>on</strong>tribute to discussi<strong>on</strong>s. Out of all resp<strong>on</strong>dents observed, a majority of resp<strong>on</strong>dents<br />

(67.6%) provide immediate alert if something goes wr<strong>on</strong>g (often or occasi<strong>on</strong>ally).<br />

No clear evidence arises from the previous findings as to whether the providers of the alerts and<br />

discussi<strong>on</strong>s c<strong>on</strong>tributors are the same customers. That is why a correlati<strong>on</strong> analysis was d<strong>on</strong>e. The<br />

correlati<strong>on</strong> was significantly substantiated.<br />

Table 2: Correlati<strong>on</strong> between providing alerts and sharing knowledge<br />

Sharing knowledge and<br />

experience, c<strong>on</strong>tribute to<br />

discussi<strong>on</strong>s<br />

answer often and<br />

occasi<strong>on</strong>ally<br />

When discovering an<br />

inappropriateness,<br />

notificati<strong>on</strong> to the<br />

company, (alert)<br />

answers often and<br />

occasi<strong>on</strong>ally<br />

Discussi<strong>on</strong><br />

c<strong>on</strong>tributors Alerts providers<br />

Pears<strong>on</strong> Correlati<strong>on</strong> 1 ,156 **<br />

Sig. (2-tailed) ,000<br />

Sum of Squares and Crossproducts<br />

143,571 28,338<br />

Covariance ,286 ,056<br />

Pears<strong>on</strong> Correlati<strong>on</strong> ,156 ** 1<br />

Sig. (2-tailed) ,000<br />

Sum of Squares and Crossproducts<br />

28,338 230,604<br />

Covariance ,056 ,459<br />

**. Correlati<strong>on</strong> is significant at the 0.01 level (2-tailed).<br />

Customers willing to provide informati<strong>on</strong> to businesses can also be characterized as follows:<br />

most of them (81%) intend to increase their purchases within a year<br />

they prefer (76%) to realize their purchases in specialized e-shops (the other opti<strong>on</strong> was universal<br />

e-shops)<br />

45% of them admit to using the internet c<strong>on</strong>necti<strong>on</strong> from their work place (for c<strong>on</strong>necti<strong>on</strong> with<br />

internet shops)<br />

there is an active populati<strong>on</strong> in the age range 20-25 exceeding the average number of<br />

c<strong>on</strong>tributors in the age group<br />

311


Kamila Tislerova<br />

6. Findings <strong>on</strong> the significance of informati<strong>on</strong> provided to customers by<br />

business<br />

There was a set of 16 questi<strong>on</strong>s relating to the type of informati<strong>on</strong> provided by business, including the<br />

form of presentati<strong>on</strong>. The importance of each piece of informati<strong>on</strong> was measured according to the<br />

scale. Using Factor analysis (factor reducti<strong>on</strong>), the following factors were determined:<br />

User-friendly way of informati<strong>on</strong> presentati<strong>on</strong> (28%)<br />

Presence of additi<strong>on</strong>al and explanatory informati<strong>on</strong> (16%)<br />

Informati<strong>on</strong> from an “independent source” such as other customers references (14%)<br />

Informati<strong>on</strong> sorted by users, not by technical features of products (13%)<br />

These factors are able to explain the 71% of the customers’ requirements c<strong>on</strong>cerning informati<strong>on</strong><br />

provided by businesses.<br />

On the other side, the less demanded type of informati<strong>on</strong> was the presence of Frequently Asked<br />

Questi<strong>on</strong>s and info c<strong>on</strong>cerning the company history (and other self-presenting data).<br />

The highest deviati<strong>on</strong>s occur in the questi<strong>on</strong> as to how important the previous purchase display is for<br />

the customer.<br />

7. C<strong>on</strong>clusi<strong>on</strong>s<br />

Even though after-purchase c<strong>on</strong>tact has become very frequent and seems to be annoying for the<br />

customers, this survey dem<strong>on</strong>strates the fact that in 45% of the cases observed, the reacti<strong>on</strong> with<br />

regard to this request for feedback is positive. About 35% is able to tolerate it and act in accordance,<br />

depending <strong>on</strong> the situati<strong>on</strong>. Only about 20% of participants have a negative attitude (of this number,<br />

just 7 % str<strong>on</strong>gly rejects this type of c<strong>on</strong>tact). The suggesti<strong>on</strong> of this paper for companies is not to<br />

doubt whether to ask for feedback or not; the questi<strong>on</strong> is how to design the request for feedback so<br />

that it is as pleasant and comfortable for the customer as possible.<br />

Also a relatively high percentage of actively resp<strong>on</strong>ding customers was found to exist. It is a matter of<br />

Customer Relati<strong>on</strong>ship <strong>Management</strong> to reward these customers, to involve them in loyalty<br />

programmes and provide some advantages and incentives to promote this kind of activity. Customer<br />

potential in informati<strong>on</strong> interacti<strong>on</strong> does exist to a significant degree.<br />

Not <strong>on</strong>ly regular feedback, but also some alerts or the sharing of knowledge and experience bel<strong>on</strong>gs<br />

to important informati<strong>on</strong> interacti<strong>on</strong>. A correlati<strong>on</strong> of groups providing alerts and discussi<strong>on</strong><br />

c<strong>on</strong>tributi<strong>on</strong> was shown to exist and some specificati<strong>on</strong>s were outlined.<br />

For the explanati<strong>on</strong> of more than 70%, four main factors (groups) of informati<strong>on</strong> provided by business<br />

were derived. Successful business should present their informati<strong>on</strong> in a very accessible way, should<br />

accompany the main info with additi<strong>on</strong>al notices and advice which might be helpful for customers.<br />

Informati<strong>on</strong> demanding too much effort <strong>on</strong> the part of the customers (like Frequently Asked Questi<strong>on</strong>s<br />

secti<strong>on</strong>) or exhibiting low value for the customers should be minimised if efficient informati<strong>on</strong><br />

interacti<strong>on</strong> is to be established.<br />

References<br />

Acs, Z. J., Arenius, P., Hay, M. and Mininiti, M. (2004). Global entrepreneurship m<strong>on</strong>itor: 2004 Executive report.<br />

MA: Babs<strong>on</strong> College and UK: L<strong>on</strong>d<strong>on</strong> Business School<br />

Daniel, E.M. and Wils<strong>on</strong>, H.N. (2003) “The role of dynamic capabilities in e-business transformati<strong>on</strong>“, <str<strong>on</strong>g>European</str<strong>on</strong>g><br />

Journal of Informati<strong>on</strong> Systems, Vol. 12, pp. 282–296.<br />

Dědková, J. (2010) Analýza nákupního chování v česko-německé části Euroregi<strong>on</strong>u Neisse-Nisa-Nysa. 1. vyd.<br />

Liberec: Technická univerzita v Liberci, ISBN 9788073725938.<br />

Eurostat. (2011). Internet purchases by individuals. [<strong>on</strong>line]. Retrieved April 15, 2012, from<br />

http://appsso.eurostat.ec.europa.eu/nui/show.do?dataset=isoc_ec_ibuy&lang=en<br />

Grant, R. Clarke, R.J. and Kyriazis, E. (2007) “A review of factors affecting <strong>on</strong>line c<strong>on</strong>sumer search behaviour<br />

from an informati<strong>on</strong> value perspective”, Journal of Marketing <strong>Management</strong>, Vol. 23, No. 5/6, pp. 519–533.<br />

Kathuria, R. and Joshi, M.P. (2007) “Envir<strong>on</strong>mental influences <strong>on</strong> corporate entrepreneurship: Executive<br />

perspectives <strong>on</strong> the internet“, Internati<strong>on</strong>al Entrepreneurship and <strong>Management</strong> Journal, Vol. 3, No. 2,<br />

pp.127-144.<br />

Kotler, P. (2004) Marketing <strong>Management</strong>. 11 editi<strong>on</strong>, Prentice Hall: New Jersey<br />

312


Kamila Tislerova<br />

Mo<strong>on</strong>, B.J. (2004) “C<strong>on</strong>sumer adopti<strong>on</strong> of the internet as an informati<strong>on</strong> search and product purchase channel:<br />

some research hypotheses”, Internati<strong>on</strong>al Journal of Internet Marketing and Advertising, Vol. 1, No. 1, pp.<br />

104–118.<br />

OECD. (2011). Guide to measuring the informati<strong>on</strong> society. doi: 10.1787/9789264113541-en<br />

OECD. (2008). Measuring the Impacts of ICT Using Official Statistics. doi:10.1787/230662252525<br />

Payne, A.F., Storbacka, K. and Frow P. (2008) “Managing the co-creati<strong>on</strong> of value”, Journal of the Academy of<br />

Marketing Science, Vol. 36, No. 4, pp. 83–96.<br />

Puccinelli, M. et al., (2009) “Customer Experience <strong>Management</strong> in Retailing: Understanding the Buying Process”,<br />

Journal of Retailing, Vol. 85, No. 1, pp. 15–30.<br />

Roztocki, N. Weistroffer, H.R. (2009) “Research Trends in Informati<strong>on</strong> and Communicati<strong>on</strong>s Technology in<br />

Developing, Emerging and Transiti<strong>on</strong> Ec<strong>on</strong>omies“.Collegium of Ec<strong>on</strong>omic Analysis, Vol. 20, pp. 113-127.<br />

Rydvalová P., Maršíková K. (2011) “Small and Medium Enterprises in the Czech Republic.” 1. Ed., Liberec:<br />

VÚTS, a.s.<br />

SIMOVÁ, J. (2010) “Internati<strong>on</strong>alizati<strong>on</strong> in the Process of the Czech Retail Development”. E+M Ek<strong>on</strong>omie a<br />

<strong>Management</strong>. Vol.. 13, No. 2., pp 78 – 91.<br />

Teo, T.S.H., Wang, P. and Le<strong>on</strong>g H.C. (2004) “Understanding <strong>on</strong>line shopping behavior using a transacti<strong>on</strong> cost<br />

ec<strong>on</strong>omics approach”, Internati<strong>on</strong>al Journal of Internet Marketing and Advertising, Vol. 1, No. 1, pp. 62–84.<br />

Tišlerová, K. (2011) “Customer´s Profitability: Methodological Approaches” Liberec Ec<strong>on</strong>omic Forum, pp 497-<br />

506.<br />

UNCTAD. (2011). Measuring the impacts of informati<strong>on</strong> and communicati<strong>on</strong> technology for development. [<strong>on</strong>line].<br />

Retrieved April, 2, 2012, from http://www.unctad.org/en/docs/dtlstict2011d1_en.pdf<br />

Žambochová, M. (2008) “Data Mining Methods with Trees”, E+M – Ek<strong>on</strong>omika a <strong>Management</strong>, Vol.9, No.1, pp<br />

126 – 132.<br />

313


Designing High Quality ICT for Altered Envir<strong>on</strong>mental<br />

C<strong>on</strong>diti<strong>on</strong>s<br />

Daryoush Daniel Vaziri, Dirk Schreiber and Andreas Gadatsch<br />

B<strong>on</strong>n-Rhine-Sieg University of Applied Science, Sankt Augustin, Germany<br />

Daryoush.Vaziri@h-brs.de<br />

Dirk.Schreiber@h-brs.de<br />

Andreas.Gadatsch@h-brs.de<br />

Abstract: This article c<strong>on</strong>cerns the design and development of Informati<strong>on</strong>- and Communicati<strong>on</strong> Technology, in<br />

particular computer systems in regard to the demographic transiti<strong>on</strong> which will influence user capabilities. It is<br />

questi<strong>on</strong>able if current applied computer systems are able to meet the requirements of altered user groups with<br />

diversified capabilities. Such an enquiry is necessary based <strong>on</strong> actual forecasts leading to the assumpti<strong>on</strong> that<br />

the average age of employees in enterprises will increase significantly within the next 50-60 years, while the<br />

percentage of computer aided business tasks, operated by human individuals, rises from year to year. This<br />

progress will precipitate specific c<strong>on</strong>sequences for enterprises regarding the design and applicati<strong>on</strong> of computer<br />

systems. If computer systems are not adapted to altered user requirements, efficient and productive utilisati<strong>on</strong><br />

could be negatively influenced. These c<strong>on</strong>sequences c<strong>on</strong>stitute the motivati<strong>on</strong> to extend traditi<strong>on</strong>al design<br />

methodologies and thereby ensure the applicati<strong>on</strong> of computer systems that are usable, independent of user<br />

capabilities. In theory as well as in practice several design and development c<strong>on</strong>cepts described are respectively<br />

applied. However, in most cases these c<strong>on</strong>cepts are c<strong>on</strong>sidered as solitary independent soluti<strong>on</strong>s. Generally,<br />

theories c<strong>on</strong>trast usability and accessibility as two different c<strong>on</strong>cepts. While the first provides possibilities for<br />

specific user groups to accomplish tasks efficiently, effectively and satisfactorily, the latter provides soluti<strong>on</strong>s<br />

taking into c<strong>on</strong>siderati<strong>on</strong> people with a wide range of capabilities, such as disabled people or people with an<br />

enduring health problem. Both c<strong>on</strong>cepts are quite extensive. Therefore developers tend to decide between these<br />

c<strong>on</strong>cepts, which always leads to failures. This article seeks to provide a universal design and development<br />

approach for computer systems, by combining these individually c<strong>on</strong>sidered c<strong>on</strong>cepts into <strong>on</strong>e comm<strong>on</strong><br />

approach. This approach will not distinguish between user groups, but instead, will provide procedures and<br />

soluti<strong>on</strong>s to design computer systems, which c<strong>on</strong>sider all relevant user capabilities. The results of this article<br />

provide a theoretical approach for design and development cycles. Enterprises will be sensitised for the<br />

identificati<strong>on</strong> of relevant user requirements and the design of human-centred computer systems.<br />

Keywords: universal design, usability, accessibility, informati<strong>on</strong> and communicati<strong>on</strong> technology, computer<br />

system, demographic transiti<strong>on</strong><br />

1. Introducti<strong>on</strong><br />

The effective and productive applicati<strong>on</strong> of computer systems is highly dependent <strong>on</strong> the user’s<br />

capabilities. Therefore it is crucial to analyse the user’s behaviour when interacting with computer<br />

systems. Figure 1 defines the authors’ comprehensi<strong>on</strong> of computer systems in the c<strong>on</strong>text of this<br />

article.<br />

user requirements<br />

hardware, software,<br />

peripheral devices<br />

computer system<br />

diverse user<br />

capabilities<br />

efficiency,<br />

productivity<br />

Figure 1: Computer system<br />

However, in many cases the requirement analysis c<strong>on</strong>centrates <strong>on</strong> specific stakeholders that are<br />

currently employed or involved with the system and as such does not c<strong>on</strong>sider requirements of<br />

potential users with divergent capabilities. Such a combinati<strong>on</strong> of system functi<strong>on</strong>alities and new user<br />

capabilities will eventually result in a mismatch that might reduce efficiency and productivity.<br />

314


2. Background<br />

Daryoush Daniel Vaziri, Dirk Schreiber and Andreas Gadatsch<br />

Figure 2 schematically visualises the mismatch of user requirements and system functi<strong>on</strong>alities.<br />

Diversity of user<br />

capabilities<br />

Mismatch of user requirements<br />

and system functi<strong>on</strong>alities<br />

Computer<br />

system lifecycle<br />

Figure 2: Coherence of user capabilities and system lifecycle<br />

The trigger for this development is the demographic transiti<strong>on</strong> of industrialised nati<strong>on</strong>s. In most<br />

industrialised countries the populati<strong>on</strong> declines and grows old (Lutz et al, 2011). Figure 3 provides an<br />

overview of the latest populati<strong>on</strong> data and estimati<strong>on</strong>s up to the year 2050 for the nati<strong>on</strong>s France,<br />

Germany and United States of America (United Nati<strong>on</strong>s, 2010).<br />

14,0<br />

11,2<br />

12,4 12,511,5<br />

9,9<br />

% aged 15‐24 in<br />

2010<br />

Demographic transiti<strong>on</strong> in industrialised countries<br />

(2010 to 2050)<br />

% aged 15‐24 in<br />

2050<br />

26,0<br />

Germany USA France<br />

18,4<br />

23,0<br />

% aged 60 or<br />

over in 2010<br />

37,5<br />

30,5<br />

26,6<br />

% aged 60 or<br />

over in 2050<br />

20,4<br />

16,8<br />

13,1<br />

% aged 65 or<br />

over in 2010<br />

30,9<br />

24,9<br />

21,2<br />

% aged 65 or<br />

over in 2050<br />

Figure 3: Demographic transiti<strong>on</strong> of industrialised countries<br />

While the percentage of people aged 15-24 declines and respectively stagnates, the percentage of<br />

people aged 60-65 or over increases significantly. As fewer young people join the labour market,<br />

enterprises need to compensate by employing older pers<strong>on</strong>nel for the required human resources. In<br />

additi<strong>on</strong>, the intergenerati<strong>on</strong>al c<strong>on</strong>tract demands that employees retire at a later date (Sanders<strong>on</strong> et<br />

al, 2010), as life expectancy rates increase (Le<strong>on</strong>, 2011). The authors identified three c<strong>on</strong>sequences,<br />

which will affect the development of computer systems.<br />

C<strong>on</strong>sequence 1: The age distributi<strong>on</strong> in enterprises will rise; therefore capabilities of older user<br />

groups might differ significantly from younger user groups. Susceptibility to disorders or injuries<br />

related to computer work tendentially has to be classified higher than for younger employees. A<br />

survey from the year 2001 examined upper extremity disorders of 485 people. The mean age of that<br />

group was 38.5 years. Seventy per cent were computer users. Significant findings of that survey<br />

included postural misalignment with protracted shoulders (78%), head forward positi<strong>on</strong> (71%),<br />

neurogenic thoracic outlet syndrome (70%) and many more (Pascarelli et al, 2001).<br />

315


Daryoush Daniel Vaziri, Dirk Schreiber and Andreas Gadatsch<br />

C<strong>on</strong>sequence 2: Enterprises will lack young qualified pers<strong>on</strong>nel that cannot be compensated for by<br />

elder employees. To sustain capability to compete <strong>on</strong> the global market, enterprises need a specific<br />

number of young employees, who may have just completed academic and/or vocati<strong>on</strong>al training to<br />

integrate modern, unprejudiced and open minds. Enterprises thereby benefit from radical, innovative<br />

ideas and therefore need to look out for additi<strong>on</strong>al sources of human capital. One source could be<br />

found in people with disabilities or people who mainly have an enduring health problem. Europe is<br />

inhabited by approx. 502 milli<strong>on</strong> people (Marcu, 2011). In 2010 approx. 67 per cent respectively 336.4<br />

milli<strong>on</strong> of Europe inhabitants were declared as working age populati<strong>on</strong> (<str<strong>on</strong>g>European</str<strong>on</strong>g> Comissi<strong>on</strong><br />

Eurostat, 2011). About 45 milli<strong>on</strong> people of the declared working age populati<strong>on</strong> either had a disability<br />

or an enduring health problem (<str<strong>on</strong>g>European</str<strong>on</strong>g> Comissi<strong>on</strong> Eurostat, 2003). Worldwide the number of<br />

disabled people aged 15 and older is estimated as 720 milli<strong>on</strong> (World Health Organizati<strong>on</strong>, 2011).<br />

However, current computer systems do not meet the requirements of this user group.<br />

C<strong>on</strong>sequence 3: Susceptibility to mental disorders induced by computer-related stress factors<br />

increases. In the latest report, Wittchen et al. learned that almost 165 milli<strong>on</strong> <str<strong>on</strong>g>European</str<strong>on</strong>g>s suffer from<br />

brain disorders like depressi<strong>on</strong>, anxiety, insomnia or dementia every year (Wittchen et al, 2010). This<br />

is an increase of about 100 milli<strong>on</strong> people compared to a similar major <str<strong>on</strong>g>European</str<strong>on</strong>g> study of brain<br />

disorders c<strong>on</strong>ducted in the year 2005 (Walker, 2011).<br />

Current HCI research does not sufficiently deal with these developments. In the professi<strong>on</strong>al c<strong>on</strong>text,<br />

most human-computer-interacti<strong>on</strong>s are still executed with keyboard or mouse devices. Graphical User<br />

Interfaces (GUI) become more complex and require the user to apply more cognitive resources. With<br />

regard to the demographic transiti<strong>on</strong>, the authors identified the need to provide a universal design<br />

approach for computer systems that takes envir<strong>on</strong>mental alterati<strong>on</strong>s into account.<br />

3. Universal approach to the design of computer systems<br />

The following paragraphs will introduce the reader to a universal design approach. C<strong>on</strong>siderati<strong>on</strong> of<br />

recommendati<strong>on</strong>s and thoughts given in these paragraphs will improve the l<strong>on</strong>g-term usability of<br />

computer systems.<br />

3.1 Human-centred design of interacti<strong>on</strong>s<br />

When it comes to defining interacti<strong>on</strong> design, three major schools of thought can be distinguished<br />

(Saffer, 2010):<br />

A technology centred view<br />

A behaviourist view<br />

A social Interacti<strong>on</strong> Design view<br />

Human-centred design can be classified in the behaviourist view. The behaviour of people using<br />

products is the central focus of interest. Rather than centred <strong>on</strong> the restricti<strong>on</strong>s of end user<br />

capabilities, the majority of professi<strong>on</strong>al system development is c<strong>on</strong>strained to specific software<br />

packages or technologies and the capabilities associated with them (Poslad, 2009; Kolko, 2011). In<br />

most cases, this leads to company cultures that are str<strong>on</strong>gly computing centred (Kolko, 2011).<br />

However, research studies showed that comprehensi<strong>on</strong> and c<strong>on</strong>siderati<strong>on</strong> of human behaviour and<br />

human capabilities, for the purposes of system development, result in more usable and accessible<br />

products (Kolko, 2011; Wickens et al, 2000). A cyclic process of perceiving, thinking, recognising,<br />

acting and evaluating acti<strong>on</strong>s can be observed, when users interact with computer systems (M<strong>on</strong>k,<br />

1998). Figure 4 shows how main elements of cogniti<strong>on</strong> interact with <strong>on</strong>e another and in the wider<br />

c<strong>on</strong>text of cognitive processing (Persad et al, 2007).<br />

Step 1: The user gets in c<strong>on</strong>tact with external stimuli, for example a GUI.<br />

Step 2: The percepti<strong>on</strong> comp<strong>on</strong>ent analyses and processes the incoming sensory informati<strong>on</strong>.<br />

Step 3: The working memory retrieves l<strong>on</strong>g-term memories and decides to act <strong>on</strong> the selected<br />

stimuli. The attenti<strong>on</strong> resources are directed to focus <strong>on</strong> the most informative parts of the stimuli<br />

and initiate acti<strong>on</strong>s and reas<strong>on</strong>ing (Mieczakowski et al, 2010).<br />

Step 4: For matching the selected stimuli with objects of similar physical properties and functi<strong>on</strong>al<br />

attributes and for grouping them into categories in memory, working memory frequently has to<br />

refer to l<strong>on</strong>g-term memory (Miller, 1956).<br />

316


Daryoush Daniel Vaziri, Dirk Schreiber and Andreas Gadatsch<br />

Step 5: If the user has experienced the stimuli before and is familiar with the GUI or computer<br />

system, informati<strong>on</strong> about them will probably affect the speed and efficiency of cognitive<br />

processing.<br />

Step 6: The user executes an acti<strong>on</strong> based up<strong>on</strong> the previous cognitive processing.<br />

1<br />

Input<br />

Low-level senses<br />

Percepti<strong>on</strong><br />

2<br />

L<strong>on</strong>g-term Memory<br />

3<br />

Working memory<br />

Executive functi<strong>on</strong><br />

Attenti<strong>on</strong>al resources<br />

Envir<strong>on</strong>ment and product<br />

5<br />

4<br />

Similarity<br />

matching<br />

6<br />

Output<br />

Acti<strong>on</strong><br />

Figure 4: Simplified model of cogniti<strong>on</strong> processing<br />

Studies that ageing and certain impairments or disabilities can have significant effects <strong>on</strong> the<br />

elements of cogniti<strong>on</strong> are depicted in figure 6 (Rabbitt 1993; Freudenthal, 1999).<br />

3.1.1 Design of graphical user interfaces<br />

The successful design of GUIs follows principles of usability engineering. Usability is defined as:<br />

“The extent to which a product can be used by specified users to achieve specified goals<br />

with effectiveness, efficiency, and satisfacti<strong>on</strong> in a specified c<strong>on</strong>text of use” (ISO, 1998).<br />

To provide an overview of the huge extent of usability engineering, figure 5 will illustrate major<br />

usability categories (Bailey et al, 2003).<br />

Page Layout<br />

Scrolling<br />

and paging<br />

Figure 5: Usability engineering<br />

Lists<br />

Graphics,<br />

Images and<br />

Multimedia<br />

Usability engineering<br />

C<strong>on</strong>tent<br />

organisati<strong>on</strong><br />

Accessibility<br />

Search<br />

Navigati<strong>on</strong><br />

317<br />

Hardware<br />

and Software<br />

Homepage<br />

Screenbased<br />

c<strong>on</strong>trols<br />

Headings,<br />

Titles and<br />

Labels<br />

Writing Web<br />

C<strong>on</strong>tent<br />

Text<br />

appearance


Daryoush Daniel Vaziri, Dirk Schreiber and Andreas Gadatsch<br />

Usability guidelines often c<strong>on</strong>sider accessibility as a detached usability category that exists al<strong>on</strong>gside<br />

the extensive amount of remaining categories. Developers are partially overwhelmed by the mass of<br />

usability principles, so that they tend to avoid the applicati<strong>on</strong> of accessibility principles. This<br />

perspective leaves the impressi<strong>on</strong> that accessibility <strong>on</strong>ly provides benefits for specific user minorities.<br />

In fact, accessibility can be c<strong>on</strong>sidered as a distinct engineering discipline that, up<strong>on</strong> applicati<strong>on</strong>,<br />

provides significant benefits for every system user. The definiti<strong>on</strong> for accessibility given by the ISO is<br />

as follows.<br />

“The usability of a product, service, envir<strong>on</strong>ment or facility by people with the widest<br />

range of capabilities” (ISO, 2008).<br />

This definiti<strong>on</strong> implies that true usability can <strong>on</strong>ly be achieved by applying accessibility principles<br />

within each usability category. The World Wide Web C<strong>on</strong>sortium (W3C) already provides an extensive<br />

guideline <strong>on</strong> Web c<strong>on</strong>tent accessibility. The following paragraphs will shortly explain each accessibility<br />

category.<br />

Perceptibility: Perceptibility implies that c<strong>on</strong>tent presented <strong>on</strong> a website or within an applicati<strong>on</strong> is<br />

perceivable for every user regardless of his capability (Pühretmair et al, 2005). To fulfil this principle,<br />

developers can integrate additi<strong>on</strong>al functi<strong>on</strong>alities like scalability or the two-channel principle. The<br />

latter is used to provide multiple opportunities for the user to succeed in a specific task (Wegge et al,<br />

2007). Furthermore the c<strong>on</strong>tent can be enriched by alternative tags, which furnish n<strong>on</strong>-textual c<strong>on</strong>tent<br />

with descripti<strong>on</strong>s. Another crucial, often underestimated, criteri<strong>on</strong> is the colour c<strong>on</strong>trast of c<strong>on</strong>tent.<br />

Depending <strong>on</strong> the combinati<strong>on</strong> of colours, viewing c<strong>on</strong>tent <strong>on</strong> the computer screen can be more than<br />

exhausting for users. Additi<strong>on</strong>ally visually-impaired users are not able to perceive specific colour<br />

combinati<strong>on</strong>s; therefore c<strong>on</strong>veyance of informati<strong>on</strong> should not be solely executed by colour changes.<br />

There are approximately 200 milli<strong>on</strong> people afflicted with dyschromatopsia worldwide and such<br />

people are not able to differentiate between red and green c<strong>on</strong>tent. An example would be to imagine<br />

high level executives, afflicted with dyschromatopsia, reviewing operating numbers that are<br />

represented in a usual traffic light-system.<br />

Understandability: This criteri<strong>on</strong> intends to make text c<strong>on</strong>tent readable and understandable as well as<br />

to make the applicati<strong>on</strong> processes appear predictable and operable. Therefore developers can for<br />

example programmatically determine the default language of the applicati<strong>on</strong> or label unusual words<br />

and abbreviati<strong>on</strong>s. A Screen-Reader, used by many disabled people to read the c<strong>on</strong>tent presented <strong>on</strong><br />

a website or within an applicati<strong>on</strong>, can <strong>on</strong>ly help the user when text or text parts are labelled correctly.<br />

To render applicati<strong>on</strong> processes predictable to the user it is an advantage to highlight any focused<br />

comp<strong>on</strong>ents. Furthermore navigati<strong>on</strong>al mechanisms that are repeated <strong>on</strong> multiple web pages or<br />

applicati<strong>on</strong> screens should be c<strong>on</strong>sistent if possible and comp<strong>on</strong>ents that share the same functi<strong>on</strong>ality<br />

should use a similar identifier like a symbol or name (W3C, 2008).<br />

Operability: To make applicati<strong>on</strong>s operable for people with disabilities or restricti<strong>on</strong>s, all functi<strong>on</strong>alities<br />

should be triggerable through a keyboard. Some people afflicted by physical movement disabilities<br />

are not able to use a computer mouse and the <strong>on</strong>ly way for them to navigate through the c<strong>on</strong>tent of a<br />

web site or applicati<strong>on</strong> is to use the tabulator-key of the keyboard. Focus order and focus visibility are<br />

important and have to be c<strong>on</strong>sidered. Hence, developers should avoid keyboard traps, which would<br />

kill the operability at a stroke. Generally the user should be provided with enough time to use, read<br />

and process the c<strong>on</strong>tent. Seizure disorders also have to be taken into account when developing a<br />

website or applicati<strong>on</strong>. So, rapidly flashing c<strong>on</strong>tent should be avoided (W3C, 2008).<br />

Technical robustness and technical openness: Website or applicati<strong>on</strong> c<strong>on</strong>tent must be robust enough,<br />

so that a variety of assisting technologies can interpret the c<strong>on</strong>tent reliably. Assisting technologies<br />

help users with restricted capabilities to perceive, understand and operate the c<strong>on</strong>tent. Screen<br />

readers or screen magnifiers are examples of assisting technologies; however, compatibility of current<br />

and future technologies has to be ensured. In additi<strong>on</strong>, fulfilling this principle prohibits redundant data<br />

and multiple versi<strong>on</strong>s. Special features of robust c<strong>on</strong>tent are listed below (W3C, 2008):<br />

Elements have complete start and end tags<br />

Elements are nested according to their specificati<strong>on</strong>s<br />

Elements do not c<strong>on</strong>tain duplicate attributes<br />

Any IDs are unique (specific excepti<strong>on</strong>s allowed)<br />

318


Daryoush Daniel Vaziri, Dirk Schreiber and Andreas Gadatsch<br />

The four-level structure elaborated by the W3C provides a robust categorisati<strong>on</strong> to integrate usability<br />

and accessibility principles. Therefore, the authors recommend the complete and equivalent<br />

transformati<strong>on</strong> of accessibility and usability principles into a corporate framework for the humancentred<br />

design of GUIs, as depicted in figure 6. Within each category the developers will find<br />

accepted criteria to ensure the accessibility and usability of GUIs.<br />

Operability<br />

Page Layout<br />

Scrolling and<br />

paging<br />

Lists<br />

Human-centred Design<br />

Graphics,<br />

Images and<br />

Multimedia<br />

Effectiveness<br />

Perceptibility<br />

Search<br />

C<strong>on</strong>tent<br />

organisati<strong>on</strong><br />

Navigati<strong>on</strong><br />

Hardware<br />

and Software<br />

Homepage<br />

Screen-based<br />

c<strong>on</strong>trols<br />

Headings,<br />

Titles and<br />

Labels<br />

Technical robustness and openness<br />

Writing Web<br />

C<strong>on</strong>tent<br />

Text<br />

appearance<br />

Efficiency Satisfacti<strong>on</strong><br />

Figure 6: Universal design approach to the development of GUIs<br />

3.1.2 Design of computer c<strong>on</strong>trol systems<br />

As menti<strong>on</strong>ed in secti<strong>on</strong> 1, studies revealed that current computer c<strong>on</strong>trol systems are mainly<br />

resp<strong>on</strong>sible for several injuries, induced by computer-aided work and therefore future envir<strong>on</strong>mental<br />

changes will call for innovative c<strong>on</strong>trol systems, which take alterati<strong>on</strong>s of human capabilities into<br />

account.<br />

The last decade has already introduced an innovative computer c<strong>on</strong>trol system. With the release of<br />

Apple’s iPh<strong>on</strong>e, the first touchscreen c<strong>on</strong>trol system became famous and affordable for the broad<br />

majority allowing visually impaired people especially to benefit from this c<strong>on</strong>trol system. In<br />

combinati<strong>on</strong> with the integrated screen reader “VoiceOver”, these minorities were able to experience<br />

a new standard of living. Automatic speech recogniti<strong>on</strong> (ASR) is another promising computer c<strong>on</strong>trol<br />

system, which is less disseminated. ASR systems provide new opportunities for the human-centred<br />

design of interacti<strong>on</strong>s, especially in the c<strong>on</strong>text of envir<strong>on</strong>mental changes, which were described in<br />

secti<strong>on</strong> 1. Theory defines specific requirements that must be fulfilled by the ASR system, in order to<br />

work properly (Marine et al, 2011). The system needs to support a framework, managing the<br />

interacti<strong>on</strong> between human and machine which includes processing of inputs and outputs that enable<br />

the user an individualized interacti<strong>on</strong> that is most natural to him and fit the skills and physical needs of<br />

the user. Rule-based systems are able to realize this requirement, as they describe the behaviour of<br />

the user in a way that the system can understand and save it. Furthermore, the user can edit and<br />

parameterize the described behaviour to fit it to his needs (Marine et al, 2011). As the intended<br />

system behaviour depends <strong>on</strong> the current system state and the c<strong>on</strong>text of the user, the system needs<br />

to permit saving, reading and changing of the current c<strong>on</strong>text (Marine et al, 2011). In practice, ASR<br />

systems are not widely disseminated am<strong>on</strong>g enterprises and this is presumably due to the fact that<br />

the accuracy rate of these systems is not 100 per cent (Freitas, 2009). The average accuracy rate lies<br />

319<br />

Understandability


Daryoush Daniel Vaziri, Dirk Schreiber and Andreas Gadatsch<br />

between 90-98 per cent, depending <strong>on</strong> software and testing envir<strong>on</strong>ment (Karpov et al, 2008; Blanc et<br />

al, 2009; Yuschik, 2010). This means that out of a hundred words spoken, 2 to 10 words would not be<br />

recognized correctly by the ASR system and according to the c<strong>on</strong>text of use, this failure rate would be<br />

unacceptable. The following example shown in figure 7 illustrates the authors assuming the<br />

applicati<strong>on</strong> of an ASR system within a SAP GUI.<br />

Figure 7: SAP GUI<br />

A major problem in using ASR technology in SAP will be the accurate executi<strong>on</strong> of specific functi<strong>on</strong>s.<br />

If the user wants to select the tab “vendor” via speech command, the system will be c<strong>on</strong>fr<strong>on</strong>ted with<br />

two identically named objects. The failure probability would be 50 per cent in this case. The more<br />

identically or similarly named objects a GUI comprises, the higher will be the failure probability. To<br />

improve voice recogniti<strong>on</strong> accuracy rate, the ASR system functi<strong>on</strong>alities could be extended by<br />

human-eye capabilities. The human-eye is able to precisely focus a desired object <strong>on</strong> a GUI. Around<br />

a focused area an acceptance radius could be defined. The user’s speech command will be matched<br />

with the objects within this radius. This might significantly diminish failure probability and allow<br />

accuracy rates between 99 and 100 per cent. Available technologies like eye tracking systems can be<br />

applied to identify the human-eye and to determine the eye focus. Figure 8 illustrates the SAP GUI<br />

from figure 9 with the c<strong>on</strong>ceptual idea of combining ASR and eye tracking technology. In the<br />

dem<strong>on</strong>strated SAP GUI, the reader also finds objects and functi<strong>on</strong>s that are either abbreviati<strong>on</strong>s or<br />

ic<strong>on</strong>s. These objects require an alternative tag, as proposed in secti<strong>on</strong> 3.1.1 to be executed by<br />

speech commands. If the user focuses <strong>on</strong> such an object, the defined alternative tag should appear,<br />

so the user is able to execute an accurate speech command.<br />

3.2 Quantifying user experiences<br />

The adaptati<strong>on</strong> of legacy systems in accordance with design approaches as introduced in secti<strong>on</strong>s<br />

3.1.1 and 3.1.2 will burden enterprises with additi<strong>on</strong>al expenditures and risks. Resp<strong>on</strong>sible actors<br />

need to be c<strong>on</strong>vinced of benefits that will arise with the implementati<strong>on</strong> of computer systems following<br />

a human-centred design philosophy, as menti<strong>on</strong>ed in this article. To c<strong>on</strong>vince ec<strong>on</strong>omic operators,<br />

quantifiable and m<strong>on</strong>etary key performance indicators are inevitable. Clarity must exist <strong>on</strong> how the<br />

320


Daryoush Daniel Vaziri, Dirk Schreiber and Andreas Gadatsch<br />

benefits of human-centred computer systems will exceed implementati<strong>on</strong> expenditures. A well-known<br />

methodology to quantify human-computer-interacti<strong>on</strong> is usability testing. When referring to usability<br />

testing, the authors align with the definiti<strong>on</strong> of Carol M. Barnum:<br />

Speech<br />

command<br />

Acceptance radius for speech command<br />

Figure 8: SAP navigati<strong>on</strong> with ASR and eye tracking<br />

Usability testing: “The activity that focuses <strong>on</strong> observing users working with a product, performing<br />

tasks that are real and meaningful to them” (Barnum, 2011).<br />

The proposed approach to planning the usability test and quantifying user experiences is composed<br />

of six steps.<br />

1.<br />

Select strategic objectives<br />

Goals,<br />

Target groups<br />

Figure 9: Usability testing approach<br />

3.<br />

Derive key<br />

metrics<br />

2.<br />

Identify test<br />

users<br />

5.<br />

C<strong>on</strong>duct<br />

usability<br />

testing<br />

Key performance<br />

indicators<br />

User<br />

profiles<br />

Results, findings<br />

4.<br />

Determine test<br />

instruments<br />

and tasks<br />

6.<br />

Evaluate<br />

results<br />

Operati<strong>on</strong>al plan<br />

3.2.1 Select strategic objectives<br />

Strategic objectives are necessary to define goals that should be achieved by the computer system.<br />

They allow an evaluati<strong>on</strong> of results and findings gained during the usability test. To identify and<br />

321


Daryoush Daniel Vaziri, Dirk Schreiber and Andreas Gadatsch<br />

quantify need for improvement they are an inevitable artefact. Possible strategic objectives are, for<br />

example, value creati<strong>on</strong> of the system, l<strong>on</strong>g-term system stability and efficient support of business<br />

processes.<br />

At this stage the determinati<strong>on</strong> of target groups is crucial as well. To receive valuable results, it is<br />

important to know the systems end user. End users could be paying customers or employees, for<br />

example. The false determinati<strong>on</strong> of target groups will lead to an inferior selecti<strong>on</strong> of test users.<br />

3.2.2 Identify test users<br />

Based <strong>on</strong> the determined target groups from step <strong>on</strong>e, test users need to be identified and in order to<br />

achieve meaningful results, the test users should be identical with stakeholders, who have been<br />

interviewed during the requirements analysis phase. If that scenario is not possible, the<br />

characteristics of selected test users should equal the characteristics of these stakeholders. It is<br />

necessary to create user profiles for each test user, to compare user characteristics and to make<br />

potential test-user-group classificati<strong>on</strong>s. Important data for elicitati<strong>on</strong> are age, professi<strong>on</strong>, experience<br />

with the system, disabilities and impairments, satisfacti<strong>on</strong> with current system, etc. The higher the<br />

diversity of capabilities of the test users, the more valuable test results can be expected. For detailed<br />

informati<strong>on</strong> <strong>on</strong> how to identify test users and structure user profiles, the authors refer to Salvendy<br />

(2012) and Rubin et al. (2008).<br />

3.2.3 Derive key metrics<br />

Key metrics or key performance indicators (KPI) represent the most important figure for quantificati<strong>on</strong><br />

of user experiences. They make test results tangible and comparable. They allow the deducti<strong>on</strong> of<br />

recommendati<strong>on</strong>s for the adaptati<strong>on</strong> of the computer system. In practice there is a variety of different<br />

metrics for the purpose of usability testing. The authors, however, suggest focusing <strong>on</strong> a few key<br />

metrics to reduce complexity. In order to achieve the goals defined in step 1 it is necessary that the<br />

computer system meets the users’ requirements. Therefore, key metrics should base <strong>on</strong> end user<br />

requirements. Possible key metrics could be, for example, task time, error rate, user satisfacti<strong>on</strong>,<br />

clicks per task, understandability of c<strong>on</strong>tent, user stress-level, cognitive load, degree of attenti<strong>on</strong>, etc.<br />

3.2.4 Determine test instruments and tasks<br />

To measure the KPIs, corresp<strong>on</strong>ding instruments have to be applied during the usability test. Some<br />

KPIs can be measured by traditi<strong>on</strong>al observati<strong>on</strong> methodologies; however, KPIs referring to the<br />

cognitive processing of the user require special equipment and know-how. To measure cognitive<br />

processing electroencephalograms (EEG) can be applied. These instruments are able to record<br />

electrical activities al<strong>on</strong>g the scalp. As output, the EEG generates a pattern of waves, which represent<br />

brain activity. The analysis of these waves allows identifying, for example, states of stress, fatigue or<br />

c<strong>on</strong>centrati<strong>on</strong> (Sharma et al, 2010). The installati<strong>on</strong> of eye tracking technology will help to measure<br />

the degree of attenti<strong>on</strong>. Eye tracking is a widely disseminated technology for usability testing. Results<br />

can be visualised in heat diagrams for example, showing secti<strong>on</strong>s of the GUI the user paid most<br />

attenti<strong>on</strong> to. The definiti<strong>on</strong> of real and meaningful tasks is an important activity in this step. The test<br />

tasks should be a part of the user behaviour. For example, a paying customer should test order<br />

processes or community functi<strong>on</strong>alities whereas an employee should be c<strong>on</strong>fr<strong>on</strong>ted with test tasks<br />

that refer to his professi<strong>on</strong> and knowledge base. After test instruments are identified and test tasks<br />

are defined, the testing envir<strong>on</strong>ment must be prepared and the operati<strong>on</strong>al plan, including size of testuser<br />

groups, resp<strong>on</strong>sibilities for c<strong>on</strong>ducti<strong>on</strong>, time for each usability test, etc., has to be created.<br />

3.2.5 C<strong>on</strong>duct usability testing<br />

For test preparati<strong>on</strong> it would be beneficial to introduce the test users to special technologies like EEG<br />

or eye tracking. The test tasks should be explained in detail. This avoids c<strong>on</strong>fusi<strong>on</strong> of the users during<br />

the usability test, which would distort any results. The test users should be as undisturbed as<br />

possible, meaning that there is enough room between test systems. This ensures that test users are<br />

influenced by each other.<br />

3.2.6 Evaluate results<br />

The findings and results acquired from the usability test need to be analysed and evaluated to derive<br />

appropriate acti<strong>on</strong>s. Particularly EEG and Eye Tracking results require special know-how from the<br />

322


Daryoush Daniel Vaziri, Dirk Schreiber and Andreas Gadatsch<br />

analysts. EEG results can provide important informati<strong>on</strong> about the cognitive load of the user while<br />

working with the computer system. Figure 10 shows how EEG results can be interpreted (Sharma et<br />

al, 2010; Mulert et al, 2010; Hamm<strong>on</strong>d, 2006).<br />

Feature Delta Theta Alpha<br />

1<br />

Alpha<br />

2<br />

Alpha<br />

3<br />

Frequency 1.5-4 Hz 4-8 Hz 9-13 Hz 14-30 Hz<br />

Occurrance Deeper stages<br />

of sleep without<br />

dreams<br />

Stage of relaxati<strong>on</strong><br />

and meditati<strong>on</strong><br />

Interpretati<strong>on</strong> User is sleeping User is in a state of<br />

deep relaxati<strong>on</strong>.<br />

He is not<br />

c<strong>on</strong>centrating <strong>on</strong> a<br />

specific task<br />

Inattentiveness<br />

Brain is not actively<br />

engaged in mental<br />

processes<br />

User is calm and lucid.<br />

However, he is not<br />

thinking and therefore<br />

not c<strong>on</strong>centrating <strong>on</strong> a<br />

specific task<br />

Beta<br />

1<br />

Beta<br />

2<br />

Beta<br />

3<br />

Brain is actively engaged in mental<br />

processes<br />

The user is completely awake and<br />

c<strong>on</strong>centrated <strong>on</strong> a specific task<br />

Good cognitive<br />

load<br />

Stress<br />

Figure 10: Interpretati<strong>on</strong> of EEG results<br />

Oscillati<strong>on</strong>s of alpha 3 and beta 1 would represent a state of positive cognitive load, while oscillati<strong>on</strong>s<br />

above beta 1 would be an indicator for negative cognitive load. Oscillati<strong>on</strong>s of beta 3 and 4 would<br />

indicate that the user is stressed, overworked or the given task is too difficult for him. Oscillati<strong>on</strong>s<br />

below alpha 3 would indicate that the user is inattentive. Waves in the range of alpha 2 and alpha 1<br />

could be an indicator that the user’s attenti<strong>on</strong> is distracted by an element of the GUI. To collect more<br />

accurate data material, it is possible to combine EEG and eye tracking instruments.<br />

4. C<strong>on</strong>clusi<strong>on</strong><br />

This article discussed the alterati<strong>on</strong>s of envir<strong>on</strong>mental c<strong>on</strong>diti<strong>on</strong>s. The authors identified<br />

c<strong>on</strong>sequences that might influence the efficiency and productivity of current computer systems and<br />

proposed a universal design approach, which takes divergent user capabilities into account. The<br />

applicati<strong>on</strong> of human-centred design perspectives was highlighted as a critical success factor. In<br />

secti<strong>on</strong> 3.1.1 the authors introduced the reader to a universal design approach for GUIs. Afterwards,<br />

secti<strong>on</strong> 3.1.2 dealt with computer c<strong>on</strong>trol systems that would be appropriate for a universal design<br />

approach. A combinati<strong>on</strong> of ASR and eye tracking technology was introduced to the reader. Finally<br />

the article closed with an approach to quantifying the benefits of human-centred computer systems.<br />

References<br />

Bailey, R. W.; Barnum, C.; Bosley, J.; Chaparro, B.; Dumas, J.; Ivory, M. Y.; John, B.; Miller-Jacobs, H.; Koyani,<br />

S. J. (2003) Research-Based Web Design & Usability Guidelines, published by U.S. Government, available:<br />

http://www.usability.gov/guidelines/guidelines_book.pdf.<br />

Barnum, C. M. (2011) Usability Testing Essentials, published by Elsevier Inc., p. 13.<br />

Blanc, I., Vento, C. (2009) Performing with Microsoft Office 2007: Introductory, USA, p. 13.<br />

<str<strong>on</strong>g>European</str<strong>on</strong>g> Commissi<strong>on</strong> Eurostat (2003) Employment of Disabled People in Europe in 2002, ISBN 1024-4352,<br />

catalogue number: KS-NK-03-026-EN-N, cited in: Buhalis, D., V.; Eichhorn, E.; Michopoulou& G. Miller.<br />

October (2005) Accessibility Market and Stakeholder analysis – One stop shop for accessible tourism in<br />

Europe, University of Surrey, UK, available:<br />

http://www.accessibletourism.org/resources/ossate_market_analysis_public_final.pdf, p. 33.<br />

<str<strong>on</strong>g>European</str<strong>on</strong>g> Commissi<strong>on</strong> Eurostat (2011) Populati<strong>on</strong> structure and ageing, available:<br />

http://epp.eurostat.ec.europa.eu/statistics_explained/index.php/Populati<strong>on</strong>_structure_and_ageing.<br />

Freitas, J., Calado, A., Barros, M. J., Dias, M. S. (2009) Spoken language interface for mobile devices, published<br />

in: Vetulani, Z., Uszkoreit, H. (2009) Human language technology-challenges of the informati<strong>on</strong> society,<br />

Third language and technology c<strong>on</strong>ference, LTD 2007, Poland, p. 34.<br />

Freudenthal, A. (1999) The Design of Home Appliances for Young and Old C<strong>on</strong>sumers. Series Ageing and<br />

Erg<strong>on</strong>omics, part 2, PhD Thesis. Delft University Press, The Netherlands ISBN 90-407-1755-9.<br />

Hamm<strong>on</strong>d, D. C. (2006) What is Neurofeedback?, published in: Internati<strong>on</strong>al society for Neurofeedback and<br />

Research, pp. 1-11.<br />

ISO, Internati<strong>on</strong>al Organizati<strong>on</strong> for Standardizati<strong>on</strong>. (1998) ISO 9241-11, Erg<strong>on</strong>omic requirements for office work<br />

with visual display terminals (VDTs) - Part 11: Guidance <strong>on</strong> usability.<br />

ISO, Internati<strong>on</strong>al Organizati<strong>on</strong> for Standardizati<strong>on</strong>. (2008) ISO 9241-171, Erg<strong>on</strong>omics of human-system<br />

interacti<strong>on</strong> -- Part 171: Guidance <strong>on</strong> software accessibility.<br />

323<br />

Beta<br />

4


Daryoush Daniel Vaziri, Dirk Schreiber and Andreas Gadatsch<br />

Karpov, A., Carbini, S., R<strong>on</strong>zhin, A., Viallet, J. E. (2008) Two similar different speech and gestures multimodal<br />

interfaces, published in: Tzovaras, D. (2008) Multimodal user interfaces-from signals to interacti<strong>on</strong>s, Berlin<br />

Heidelberg, p. 169.<br />

Kolko, J. (2011) Thoughts <strong>on</strong> Interacti<strong>on</strong> Design, published by Elsevier Inc., pp. 13-24.<br />

Le<strong>on</strong>, D. A. March (2011) Trends in <str<strong>on</strong>g>European</str<strong>on</strong>g> life expectancy: A salutary view, published in: Internati<strong>on</strong>al journal<br />

of epidemiology 2011, pp. 1-7.<br />

Lutz. W. et al. (2011) Demographic challenges for sustainable Development, published in: Internati<strong>on</strong>al Institute<br />

for applied systems analysis (IIASA), September 30 – October 1 st 2011, available:<br />

http://www.iiasa.ac.at/Research/POP/Laxenburg%20Declarati<strong>on</strong>%20<strong>on</strong>%20Populati<strong>on</strong>%20and%20Develop<br />

ment.html.<br />

Marcu, M. (2011) Populati<strong>on</strong> grows in twenty EU Member States - Populati<strong>on</strong> change in Europe in 2010: first<br />

results, published in: Eurostat - statistics in focus, Vol. 38, available:<br />

http://epp.eurostat.ec.europa.eu/cache/ITY_OFFPUB/KS-SF-11-038/EN/KS-SF-11-038-EN.PDF, pp. 1-4.<br />

Marine, A., Stocklöw, C., Braun, A., Limberger, C., Hofmann, C., Kuijper, A. (2011) Interactive Pers<strong>on</strong>alizati<strong>on</strong> of<br />

ambient assisted living envir<strong>on</strong>ments, published in: Smith, M. J., Salvendy, G. (2011) Human Interface, Part<br />

I, HCII, LNCS 6771, Berlin / Heidelberg, p. 573.<br />

Mieczakowski, A; Langd<strong>on</strong>, P. M.; Clarks<strong>on</strong>, P. J. (2010) Investigating Designers’ Cognitive Representati<strong>on</strong>s for<br />

Inclusive Interacti<strong>on</strong> Between Products and Users, published in: Langd<strong>on</strong>, P. M.; Clarks<strong>on</strong>, P. J.; Robins<strong>on</strong>,<br />

P. (2010) Designing inclusive interacti<strong>on</strong>s, published by Springer L<strong>on</strong>d<strong>on</strong>, pp. 133-144.<br />

Miller, G. A. (1956) The magical number seven, plus or minus two: some limits <strong>on</strong> our capacity for processing<br />

informati<strong>on</strong>, published in: Psychological review, 63, pp. 81-97.<br />

M<strong>on</strong>k, A. (1998) Cyclic interacti<strong>on</strong>: a unitary approach to intenti<strong>on</strong>, acti<strong>on</strong> and the envir<strong>on</strong>ment,published in:<br />

Cogniti<strong>on</strong>, 68, pp. 95-110.<br />

Mulert, C.; Lemieux, L. (2010) EEG- FMRI: Physiology, Technique and Applicati<strong>on</strong>s, published by Springer Berlin<br />

/ L<strong>on</strong>d<strong>on</strong>..<br />

Pascarelli, E. F.; Hsu, Y. P. (2001) Understanding Work-Related Upper Extremity Disorders: Clinical Findings in<br />

485 Computer Users, Musicians, and Others, published in: Journal of Occupati<strong>on</strong>al Rehabilitati<strong>on</strong>, Vol. 11,<br />

No. 1, 2001, pp. 1-21.<br />

Persad, U.; Langd<strong>on</strong>, P.; Clarks<strong>on</strong>, P. J. (2007) Characterising user capabilities to support inclusive design<br />

evaluati<strong>on</strong>.Published in: Universal access in the informati<strong>on</strong> society.Special Issue <strong>on</strong> Designing Accessible<br />

Technology, 6, pp. 119-135.<br />

Poslad, S. (2009) Ubiquitous Computing-Smart devices, envir<strong>on</strong>ments and interacti<strong>on</strong>s, published by John Wiley<br />

& S<strong>on</strong>s Ltd.<br />

Pühretmair, F., Miesenberger, K. (2005) Making sense of accessibility in IT Design - usable accessibility vs.<br />

accessible usability, published in: IEEE Computer society, Proceedings of the 16 th internati<strong>on</strong>al Workshop<br />

<strong>on</strong> Database and Expert Systems Applicati<strong>on</strong>s (DEXA'05), 1529-4188/05, p. 2.<br />

Rabbitt, P. (1993) Does it all go together when it goes? The nineteenth Bartlett memorial lecture. The Quarterly<br />

Journal of Experimental Psychology, 46A, pp. 385-434.<br />

Rubin, J. B.; Rubin, J.; Chisnell, D. (2008) Handbook of usability testing - How to plan, design, and c<strong>on</strong>duct<br />

effective tests, 2 nd Editi<strong>on</strong>, published by Wiley publishing Inc.<br />

Saffer, D. (2010) Designing for Interacti<strong>on</strong>: Creating Innovative Applicati<strong>on</strong>s and Devices, pp. 4-15.<br />

Salvendy, G. (2012) Human Factors and Erg<strong>on</strong>omics, 4 th Editi<strong>on</strong>, published by John Wiley & S<strong>on</strong>s Inc., pp.<br />

1276-1283.<br />

Sanders<strong>on</strong>, W.; Scherbov, S. (2010) Remeasuring Aging, published in: Science, September 2010, vol. 329, No.<br />

5997, pp. 1278-1288.<br />

Sharma, J. K.; Singh, D.; Deepak, K. K.; Agrawal, D. P. (2010) Neuromarketing-A peep into customers’ minds,<br />

published by PHI learning private limited, p. 126.<br />

United Nati<strong>on</strong>s, populati<strong>on</strong> Divisi<strong>on</strong> of the Department of Ec<strong>on</strong>omic and Social Affairs of the United Nati<strong>on</strong>s<br />

Secretariat (2010) world populati<strong>on</strong> prospects, the 2010 revisi<strong>on</strong>, available:<br />

http://esa.un.org/unpd/wpp/unpp/Panel_profiles.htm.<br />

W3C. December 11, (2008) Web C<strong>on</strong>tent Accessibility Guidelines (WCAG) 2.0, available:<br />

http://www.w3.org/TR/2008/REC-WCAG20-20081211/.<br />

Walker, C. (2011) Europe plagued with mental health problems, published by: Mental Healthy, available:<br />

http://www.mentalhealthy.co.uk/news/841-europe-plagued-with-mental-health-problems.html.<br />

Wegge, K. P., Zimmermann, D. (2007) Accessibility, Usability, Safety, Erg<strong>on</strong>omics, C<strong>on</strong>cepts, Models and<br />

Differences, published in: Stephanidis, C., Universal Access in HCI, Part I, HCII 2007, LNCS 4554, Berlin /<br />

Heidelberg: Springer, p. 297.<br />

Wickens, C. D.; Hollands, J. G. (2000) Engineering psychology and human performance, 3 rd edn.Prentice Hall,<br />

Upper Saddle River, NJ, US.<br />

Wittchen, H. U., et al. (2010) The size and burden of mental disorders and other disorders of the brain in Europe<br />

2010, ECNP/EBC REPORT 2011, published in: <str<strong>on</strong>g>European</str<strong>on</strong>g> Neuropsychopharmacology, (2011) vol. 21, pp.<br />

655–679.<br />

World Health Organizati<strong>on</strong>. (2011) World report <strong>on</strong> disability, ISBN 978 92 4 068521 5, available:<br />

http://whqlibdoc.who.int/publicati<strong>on</strong>s/2011/9789240685215_eng.pdf, p. 27.<br />

Yuschik, M. (2010) Leveraging multimodality to improve call center productivity, published in: Neustein, A. 2010,<br />

Advances in speech recogniti<strong>on</strong>-Mobile envir<strong>on</strong>ments, call centers and clinics, New York, p. 143.<br />

324


An Analysis of the Problems Linked to Ec<strong>on</strong>omic<br />

Evaluati<strong>on</strong> of <strong>Management</strong> Support Informati<strong>on</strong> Systems in<br />

Poland <strong>on</strong> the Example of ERP/CRM Class Applicati<strong>on</strong>s -<br />

Problem Analysis<br />

Bartosz Wachnik<br />

Warsaw University of Technology, Faculty of Producti<strong>on</strong> Engineering, Institute<br />

of Producti<strong>on</strong> Systems Organisati<strong>on</strong>, Warsaw<br />

bartek@wachnik.eu<br />

Abstract: Research shows that the expenditure of internati<strong>on</strong>al companies <strong>on</strong> IS systems has been gradually<br />

growing. There is a comm<strong>on</strong> belief that IS development has a direct or indirect impact <strong>on</strong> the ec<strong>on</strong>omic<br />

effectiveness of a company. ERP and CRM class systems c<strong>on</strong>stitute <strong>on</strong>e of the most important groups of<br />

management support informati<strong>on</strong> systems in service and manufacturing companies. A lot of research is aimed at<br />

developing models that allow us to identify the correlati<strong>on</strong> between expenditure <strong>on</strong> ERP and CRM systems and<br />

the ec<strong>on</strong>omic benefits to a company. Nevertheless, we need to underline that the result of this research is not<br />

evident. The research is characterised by a lack of c<strong>on</strong>sensus between representatives of science, as well as<br />

professi<strong>on</strong>als. The subject matter menti<strong>on</strong>ed below is <strong>on</strong>e of the priorities facing the further development of<br />

business informatics. The author of this article presents the result of research related to the ec<strong>on</strong>omic<br />

effectiveness evaluati<strong>on</strong> in IS investments in ERP and CRM systems implementati<strong>on</strong> in Poland. The research<br />

c<strong>on</strong>ducted will help better understand the logic predominant am<strong>on</strong>gst Polish entrepreneurs and management in<br />

the realisati<strong>on</strong> of IS investments and ec<strong>on</strong>omic evaluati<strong>on</strong> of performed investments in ERP and CRM systems.<br />

Keywords: ec<strong>on</strong>omic evaluati<strong>on</strong>, effectiveness, IS investments, MIS, ERP, CRM<br />

1. Introducti<strong>on</strong><br />

ERP class systems c<strong>on</strong>stitute a group of integrated computer systems supporting complex enterprise<br />

management. They originate from an evoluti<strong>on</strong> of management support systems which evolved into<br />

ever newer and more advanced soluti<strong>on</strong>s. Currently, modern ERP systems are multi-module<br />

applicati<strong>on</strong>s c<strong>on</strong>taining a wide range of functi<strong>on</strong>alities required by a vast majority of commercial<br />

and/or manufacturing companies.<br />

CRM systems are treated as informati<strong>on</strong> systems supporting customer relati<strong>on</strong>ship management and<br />

the effective functi<strong>on</strong>ing of sales departments. We need to highlight the subtle difference between the<br />

philosophy of CRM and ERP system performance. While both of them are expected to increase the<br />

profits (and the value) of an enterprise, the fulfilment of this goal is carried out with different<br />

assumpti<strong>on</strong>s.<br />

The choice of ERP and CRM class systems for research stems from the fact that they are the most<br />

popular and predominant types of management support informati<strong>on</strong> systems in commercial and<br />

manufacturing companies. They usually c<strong>on</strong>stitute the main IT “bloodstream” of a company and their<br />

implementati<strong>on</strong> is am<strong>on</strong>gst the most complicated organisati<strong>on</strong>al projects for companies.<br />

In the early 90s, it was observed that the sole use of IT does not necessarily result in the ec<strong>on</strong>omic<br />

success of a company. Studies showed that a higher expenditure <strong>on</strong> IT ventures is not accompanied<br />

by a visibly higher productivity of employees, enterprises, branches or even whole ec<strong>on</strong>omic systems;<br />

an analysis of statistical data can often lead to a c<strong>on</strong>trary c<strong>on</strong>clusi<strong>on</strong> – indicating a lack of correlati<strong>on</strong><br />

between the level of IT expenditure and measurable productivity indicators (Dudycz H., Dyczkowski<br />

M. 2006). Therefore, an increasing number of both middle and top level managers ask themselves:<br />

How to measure ec<strong>on</strong>omic effectiveness in IS implementati<strong>on</strong> projects, including ERP and CRM class<br />

systems? How to maximise business value of investment in modern IT ?<br />

The authors of research <strong>on</strong> ec<strong>on</strong>omic effectiveness evaluati<strong>on</strong> in IS investments lack a c<strong>on</strong>sensus <strong>on</strong><br />

the adequacy of the methods used, their range and qualities. Suggesti<strong>on</strong>s <strong>on</strong> the directi<strong>on</strong> of further<br />

research depend, to a great extent, <strong>on</strong> the stand the researchers take <strong>on</strong> the difference between IS<br />

investments and other types of investment (Cypryjański J. 2007).<br />

325


Bartosz Wachnik<br />

From the perspective of IS investment ec<strong>on</strong>omic effectiveness evaluati<strong>on</strong> analysis, the most important<br />

classificati<strong>on</strong> of costs and benefits is the classificati<strong>on</strong> based <strong>on</strong> the direct link with ec<strong>on</strong>omic results<br />

of an organisati<strong>on</strong>. Looking at the link between the use of informati<strong>on</strong> systems and an organisati<strong>on</strong>’s<br />

ec<strong>on</strong>omic results, the subject literature divides them into direct and indirect ( Niedźwiedziński M.<br />

1989).<br />

There is a group of scientists and practiti<strong>on</strong>ers who highlight that investing in IS enterprises differs<br />

from other investments because costs and profits are very difficult to identify and quantify, while n<strong>on</strong>material<br />

and indirect factors may be of great importance (Powell P.L, 1999). According to them, further<br />

research should be chiefly focused <strong>on</strong> searching for new methods that will help identify and quantify<br />

n<strong>on</strong>-material, indirect costs and benefits better. The opp<strong>on</strong>ents (Weill P., Broadbent M. 1998) of this<br />

positi<strong>on</strong> believe that IS investments do not differ from other investments. They assume that the value<br />

of an IS investment depends utterly <strong>on</strong> the way this investment may render an organisati<strong>on</strong> more<br />

efficient and effective and argue that it is not necessary to create any specific parameters for IT<br />

(Remenyi D., M<strong>on</strong>ey A., Sherwood-Smith M., 2000). From the perspective of an investment’s<br />

character, this group equals IS investments, e.g. ERP or CRM investments, to investments in new<br />

producti<strong>on</strong> lines, warehouses or means of communicati<strong>on</strong> in a company. As a result, they believe that<br />

the existing methods are applicable to the same extent both in IT investments and any other type of<br />

investments and think that we should focus <strong>on</strong> researching how the existing methods could be used<br />

and perfected. The third directi<strong>on</strong> of research has been derived from an attempt to c<strong>on</strong>sider the<br />

arguments of both sides. It is directed at using the existing methods more efficiently through defining<br />

the specific character of IS investments and the rules of selecting process evaluati<strong>on</strong> methods that<br />

would be adequate to it (Cypryjański J. 2007).<br />

The author of this article aims at presenting the results of research <strong>on</strong> ec<strong>on</strong>omic evaluati<strong>on</strong> of IS<br />

investments effectiveness in ERP and CRM system implementati<strong>on</strong>. The choice of this goal and<br />

range of research has the following research c<strong>on</strong>sequences. First of all, it allows for an insight into the<br />

rarely researched subject of ec<strong>on</strong>omic evaluati<strong>on</strong> of IS investments in ERP and CRM system<br />

implementati<strong>on</strong> am<strong>on</strong>gst Polish entrepreneurs and management pers<strong>on</strong>nel. Sec<strong>on</strong>dly, it is aimed at<br />

verifying empirically the methods of ec<strong>on</strong>omic evaluati<strong>on</strong> in selected IT projects relevant to the<br />

functi<strong>on</strong>ing of an enterprise.<br />

2. Research methods<br />

The research was c<strong>on</strong>ducted between July and October 2011 and c<strong>on</strong>sisted of questi<strong>on</strong>naires. The<br />

interviewers asked questi<strong>on</strong>s to the interviewees <strong>on</strong> the teleph<strong>on</strong>e. A sample of 250 enterprises was<br />

chosen for the research, according to the following criteria:<br />

Commercial and/or manufacturing companies<br />

Between 50 and 500 employees<br />

Have their own IT department<br />

20mln zloty minimum income (the equivalent of 5 mln EUR) , 500mln (the equivalent of 125 mln<br />

EUR) zloty maximum income<br />

ERP and/or CRM implementati<strong>on</strong> at least 3 years ago<br />

There are 1 670 000 active companies in Poland. According to EU definiti<strong>on</strong> 18 000 of them are<br />

medium companies but <strong>on</strong>ly 11% of them are using ERP and CRM systems. A sample of 250<br />

enterprises is 13% of total numbers of medium companies that are using ERP, CRM systems. The<br />

companies included those with both Polish and foreign capital, widely aut<strong>on</strong>omous in terms of<br />

ERP/CRM choice and the way of performing the implantati<strong>on</strong> of an IS system. The companies came<br />

from the following provinces: Mazovian, Greater Poland, Lower Silesian, Kuyavian-Pomeranian,<br />

Lublin, Świętokrzyskie and Silesian. The structure of the companies is presented in Table 1,<br />

according to the Polish Classificati<strong>on</strong> of Activity.<br />

Percentage compositi<strong>on</strong> of incidence for respective ERP/CRM systems in the examined group is<br />

following: PeopleSoft – 1%, Siebel – 1%, Gardens – 1%, JDEdwards – 3%, GreatPlains – 3%, Impuls<br />

(Polish domestic soluti<strong>on</strong>s) – 4%, Epicor/Scala – 4%, Dynamics CRM – 4%, SAP BusinessOne –<br />

5%, Dynamics Nav – 5%, Asseco Safo (Polish domestic soluti<strong>on</strong>s) – 5%, Exact -5%, Macrologic –<br />

6%, CDN XL (Polish domestic soluti<strong>on</strong>s) – 6%, IFS – 7%, Dynamics AX – 8%, Oracle Financial – 9%,<br />

Simple – 10%, SAP R/3 – 12%. The selected companies have achieved good or average results in<br />

326


Bartosz Wachnik<br />

their branches – so they are neither leaders nor marginal companies. The aim of the research was<br />

reaching people directly or indirectly engaged in the selecti<strong>on</strong> process of an ERP/CRM informati<strong>on</strong><br />

system as well as in its implementati<strong>on</strong>. The resp<strong>on</strong>dents were company owners, directors, members<br />

of the board, financial directors or IT directors. Hence, understanding their views and cognitive maps<br />

is crucial for describing the dominant logics of operati<strong>on</strong> in performing IT projects and ec<strong>on</strong>omic<br />

evaluati<strong>on</strong> of the ERP/CRM investments. Interviews in successive companies have proved that the<br />

chosen group of resp<strong>on</strong>dents was relevant. A great majority of resp<strong>on</strong>dents had a profound<br />

knowledge of ERP/CRM system implementati<strong>on</strong> issues.<br />

Table 1: The structure of surveyed companies according to the Polish Classificati<strong>on</strong> of Activity<br />

Department Descripti<strong>on</strong> %<br />

51 Air transport 2%<br />

85 Educati<strong>on</strong> 2%<br />

88 Social care without accommodati<strong>on</strong> 2%<br />

50 Water transport<br />

Manufacture of leather and leather<br />

3%<br />

15<br />

products<br />

Services to buildings and<br />

4%<br />

81<br />

landscape activities 4%<br />

31 Manufacture of furniture<br />

Repair and maintenance of<br />

computers and pers<strong>on</strong>al and<br />

5%<br />

95<br />

household goods<br />

Other professi<strong>on</strong>al, scientific and<br />

7%<br />

74<br />

technical activities<br />

Architectural and engineering<br />

activities; technical testing and<br />

8%<br />

71<br />

analysis<br />

Manufacture of machinery and<br />

9%<br />

28<br />

equipment nec<br />

Manufacture of fabricated metal<br />

products, except machinery and<br />

12%<br />

25<br />

equipment 19%<br />

10 Manufacture of food products 23%<br />

Source: Own study.<br />

3. Research results<br />

A detailed company informatisati<strong>on</strong> strategy c<strong>on</strong>cerning an ERP/CRM system should include the<br />

reas<strong>on</strong>s for undertaking this kind of IS venture that result from organisati<strong>on</strong> strategy and its system of<br />

values, and c<strong>on</strong>sequently its l<strong>on</strong>g-term goals, as well as priorities in investment evaluati<strong>on</strong>. Hence,<br />

strategic informati<strong>on</strong> initiatives may be undertaken for <strong>on</strong>e or several reas<strong>on</strong>s.<br />

Possible reas<strong>on</strong>s for<br />

undertaking an ERP/CRM<br />

implementati<strong>on</strong> as an IS<br />

venture<br />

Survival<br />

Innovati<strong>on</strong><br />

Platforms of<br />

changes<br />

Figure 1: Possible reas<strong>on</strong>s for undertaking an ERP/CRM system implementati<strong>on</strong> decisi<strong>on</strong><br />

327


Bartosz Wachnik<br />

Source: Own study <strong>on</strong> the basis of F. Bannister, D. Remenyi, “Why IT C<strong>on</strong>tinues to Matter: Reflecti<strong>on</strong><br />

<strong>on</strong> the Strategic Value of IT”, The Electr<strong>on</strong>ic Journal Informati<strong>on</strong> Systems Evaluati<strong>on</strong>, vol. 8, iss. 3,<br />

2005, http://www.ejise.com.<br />

We can distinguish the following strategies of choosing and implementing ERP/CRM systems:<br />

Strategy linked to company’s survival <strong>on</strong> the market treats an ERP system implementati<strong>on</strong> as<br />

an instrument allowing the company to survive <strong>on</strong> the market. An example of such a strategy may<br />

be implementing an ERP system in a pharmaceutical company in the area of producti<strong>on</strong> and<br />

logistics where processes are subject to obligatory GMP/FDA quality standard requirements.<br />

Strategy linked to the need to achieve innovati<strong>on</strong> saltatorily treat an ERP system<br />

implementati<strong>on</strong> as an instrument allowing to achieve process innovati<strong>on</strong>s quickly and uniquely,<br />

resulting in, e.g., lowering costs through stock and planning producti<strong>on</strong> power. An example of this<br />

strategy may be implementing an ERP system in an aggregate mine which, <strong>on</strong> the <strong>on</strong>e hand, has<br />

a guaranteed market for its products for many years, and <strong>on</strong> the other hand would like to use a<br />

salutatory innovati<strong>on</strong> to achieve a measurable decrease of producti<strong>on</strong> costs, becoming a cost<br />

leader in its branch.<br />

Platform for changes strategy treats an ERP system implementati<strong>on</strong> as a platform for<br />

introducing permanent, step changes in the period of the enterprise’s system lifecycle. This<br />

situati<strong>on</strong> takes place in companies that functi<strong>on</strong> very dynamically <strong>on</strong> the changing market. Thus,<br />

an ERP system becomes a platform for permanent organisati<strong>on</strong>al changes which allow it to follow<br />

changing market expectati<strong>on</strong>s.<br />

Table 2 presents percentage structure of resp<strong>on</strong>dents’ answers to the questi<strong>on</strong> regarding the choice<br />

of company informatisati<strong>on</strong> strategy c<strong>on</strong>cerning ERP/CRM system and c<strong>on</strong>sequently making a<br />

decisi<strong>on</strong> about system implementati<strong>on</strong>. The resp<strong>on</strong>dents could choose more than <strong>on</strong>e answer.<br />

Table 2: Reas<strong>on</strong>s for making an ERP/CRM system implementati<strong>on</strong> decisi<strong>on</strong> – percentage structure<br />

Own study<br />

Strategies %<br />

Strategy linked to company’s survival <strong>on</strong> the<br />

68<br />

market<br />

Strategy linked to the need to achieve<br />

9<br />

innovati<strong>on</strong> saltatorily<br />

Platform for changes strategy 32<br />

Other 6<br />

Research shows that, in planning an ERP/CRM system implementati<strong>on</strong>, the resp<strong>on</strong>dents are mostly<br />

guided by company’s survival <strong>on</strong> the market strategy, for 32% of the resp<strong>on</strong>dents the reas<strong>on</strong> for<br />

undertaking an ERP/CRM system implementati<strong>on</strong> project is building a potential platform for<br />

organisati<strong>on</strong>al changes. Another 9% of the resp<strong>on</strong>dents treat this group of systems as a possibility to<br />

achieve saltatory innovati<strong>on</strong>. Table 3 presents the percentage structure of resp<strong>on</strong>dents’ answers to<br />

questi<strong>on</strong>s c<strong>on</strong>cerning the decisi<strong>on</strong> criteria in ERP/CRM class system investments. The resp<strong>on</strong>dents<br />

could choose more than <strong>on</strong>e answer.<br />

Table 3: Choice criteria in ERP/CRM class informati<strong>on</strong> system investment decisi<strong>on</strong>s – percentage<br />

structure<br />

Choice criterial %<br />

Compliance of client’s requirements with<br />

38%<br />

functi<strong>on</strong>al range of systems<br />

SLA cost 28%<br />

Purchase cost (implementati<strong>on</strong> service, licence) 19%<br />

Supplier’s references 9%<br />

Possibility of system development – scalability 8%<br />

Possibility of achieving a direct ec<strong>on</strong>omic profit,<br />

i.e. profit coming directly from the systems<br />

Possibility of achieving an indirect ec<strong>on</strong>omic<br />

profit, i.e. process automati<strong>on</strong>, an informing and<br />

organising system functi<strong>on</strong>.<br />

Source: Own study<br />

328<br />

6%<br />

2%


Bartosz Wachnik<br />

Research shows that entrepreneurs choosing an ERP/CRM system focus <strong>on</strong> questi<strong>on</strong>s relating to<br />

guaranteeing the required functi<strong>on</strong>ality by the system – 38%, the cost of purchase and<br />

implementati<strong>on</strong> - 19% and its later usage – 28%. Only 6% of resp<strong>on</strong>dents have defined a direct<br />

ec<strong>on</strong>omic profit as a choice criteri<strong>on</strong> in an ERP/CRM system selecti<strong>on</strong>, which indicates that for the<br />

majority of entrepreneurs implementing an ERP/CRM system is not syn<strong>on</strong>ymous with ec<strong>on</strong>omic profit.<br />

Only 2% of resp<strong>on</strong>dents have defined an indirect ec<strong>on</strong>omic profit as an ERP/CRM system choice<br />

criteri<strong>on</strong>. 9% of resp<strong>on</strong>dents define the supplier’s references as an ERP/CRM system choice criteri<strong>on</strong>,<br />

which c<strong>on</strong>firms that the quality of implementati<strong>on</strong> depends mostly <strong>on</strong> the competence and experience<br />

of c<strong>on</strong>sultants performing the project. There is an interesting correlati<strong>on</strong> between decisi<strong>on</strong> criteria<br />

linked to ERP/CRM class informati<strong>on</strong> system investments and the questi<strong>on</strong> of ec<strong>on</strong>omic evaluati<strong>on</strong> of<br />

company investments in this class of IS soluti<strong>on</strong>s. The subject literature (Cr<strong>on</strong>holm S., Goldkuhl G.,<br />

2003), in relati<strong>on</strong> to time, c<strong>on</strong>siders two perspectives of IS projects’ ec<strong>on</strong>omic evaluati<strong>on</strong>.<br />

The ex-ante evaluati<strong>on</strong> performed before launching a project, aimed at defining the possible<br />

future influence of its resulting IS soluti<strong>on</strong> <strong>on</strong> the company’s ec<strong>on</strong>omic situati<strong>on</strong>. The ex-ante<br />

evaluati<strong>on</strong> should be part of formal feasibility study of investment in any IT system.<br />

The ex-post evaluati<strong>on</strong> performed after the completi<strong>on</strong> of an IS project, aimed at evaluating the<br />

achieved results. The ex-post evaluati<strong>on</strong> does not c<strong>on</strong>sider the probable incidences of particular<br />

effects or the opti<strong>on</strong>ality of its course and all the risk has been discounted during its course.<br />

Table 4 presents research results for the percentage structure of the use of methods allowing defining<br />

the ec<strong>on</strong>omic aspect of ERP/CRM class informati<strong>on</strong> system implementati<strong>on</strong> investments from the exante<br />

and the ex-post perspective. Resp<strong>on</strong>dents could choose more than <strong>on</strong>e answer. Research<br />

indicates that before launching ERP/CRM class informati<strong>on</strong> system implementati<strong>on</strong> projects, 63% of<br />

chosen companies do not c<strong>on</strong>duct any ec<strong>on</strong>omic analysis of the planned investment. Within three<br />

years of completing an ERP/CRM class informati<strong>on</strong> system implementati<strong>on</strong>, 72% resp<strong>on</strong>dent<br />

companies have not c<strong>on</strong>ducted any ec<strong>on</strong>omic analysis of the performed investment that could show<br />

possible ec<strong>on</strong>omic profit or loss. Research shows that both in the ex-ante and the ex-post<br />

perspective, mostly quantitative and qualitative methods are used, i.e. ROI, Payback Period, IRR,<br />

EVA, multi-criterial methods, TCO, portfolio methods, and the presented methods are used to the<br />

same degree in case of IT and other types of investment.<br />

Table 4: Percentage structure of the methods allowing to define the ec<strong>on</strong>omic aspect of ERP/CRM<br />

class informati<strong>on</strong> system implementati<strong>on</strong> investments from the ex-ante and the ex-post<br />

perspective<br />

Ex-ante Ex-post<br />

No. Methods used % %<br />

1 ROI 10% 9%<br />

2 Payback Period 9% 7%<br />

3 IRR 8% 4%<br />

4 EVA 1% 4%<br />

5 Multi-criterial methods<br />

(strategic method) 5% 6%<br />

6 TCO 3% 3%<br />

7 Portfolio methods 2% 7%<br />

8 No measurement 63% 72%<br />

9<br />

Source: Own study.<br />

Other 8% 4%<br />

As far as similar analysis c<strong>on</strong>ducted in other countries is c<strong>on</strong>cerned, research by L.G. Paul and P.<br />

Tate [8] performed in 2002 shows that am<strong>on</strong>gst 288 financial directors in American companies over<br />

50% use the ROI method and a method defining the return <strong>on</strong> investment in informati<strong>on</strong> technology in<br />

general. Only 14% of resp<strong>on</strong>dent declared a complete lack of IS projects ec<strong>on</strong>omic evaluati<strong>on</strong><br />

measurement.<br />

Table 5: Percentage distributi<strong>on</strong> of IS projects’ ec<strong>on</strong>omic evaluati<strong>on</strong> methods used by financial<br />

directors<br />

Methods of IT ec<strong>on</strong>omic evaluati<strong>on</strong> used<br />

%<br />

by financial directors<br />

ROI 64<br />

Payback Period 63<br />

IRR 42<br />

329


Bartosz Wachnik<br />

Methods of IT ec<strong>on</strong>omic evaluati<strong>on</strong> used<br />

%<br />

by financial directors<br />

EVA 35<br />

Other 5<br />

Lack of measurement 14<br />

Source: L.G. Paul, P. Tate, CFO Mind Shift: Technology Creates Value, CFO Publishing Corporati<strong>on</strong>,<br />

Bost<strong>on</strong>, MA 2002<br />

Other research, devised by IDG Research & Getr<strong>on</strong>ics in 2002, show that am<strong>on</strong>gst 456 directors of IT<br />

departments in the USA and six <str<strong>on</strong>g>European</str<strong>on</strong>g> countries, i.e. the UK, Holland, Italy, Spain, France and<br />

Germany, 18% use the ROI method for ec<strong>on</strong>omic evaluati<strong>on</strong> of IS implementati<strong>on</strong> and 29% use the<br />

return <strong>on</strong> investment method. Only 5% of resp<strong>on</strong>dents declared a complete lack of measurement in<br />

ec<strong>on</strong>omic evaluati<strong>on</strong> of IS projects.<br />

Table 6: Percentage distributi<strong>on</strong> of IS projects’ ec<strong>on</strong>omic evaluati<strong>on</strong> methods used by IT directors<br />

Method of ec<strong>on</strong>omic evaluati<strong>on</strong> of<br />

%<br />

informati<strong>on</strong> systems implementati<strong>on</strong> used<br />

by IT departments directors<br />

Project completed within planned period<br />

50<br />

and budget<br />

Decrease of cost 49<br />

Increase of productivity 47<br />

Increase of profit/income 36<br />

Attaining flexibility and scalability 31<br />

TCO 30<br />

Payback Period 29<br />

Redukcja zasobów osobowych 22<br />

ROI 18<br />

Other 2<br />

No measurement 5<br />

Source: IDG Research & Getr<strong>on</strong>ics, The CIO Agenda: Taking Care of Business, CxO Media 2002.<br />

We need to underline that in both cases research c<strong>on</strong>ducted am<strong>on</strong>gst resp<strong>on</strong>dents from the USA and<br />

Western Europe c<strong>on</strong>cern the total of investment in informati<strong>on</strong> technology, not <strong>on</strong>ly ERP/CRM class<br />

systems and, additi<strong>on</strong>ally, it does not define the perspective of the c<strong>on</strong>ducted analysis, i.e. ex-post or<br />

ex-ante. Hence, it is not possible to compare and c<strong>on</strong>trast it from a methodical perspective with the<br />

author’s research; however, all the research c<strong>on</strong>sidered allows us to understand better the views and<br />

cognitive maps of decisi<strong>on</strong>-makers, which has a crucial importance for the descripti<strong>on</strong> of predominant<br />

acti<strong>on</strong> logics in IS projects’ realisati<strong>on</strong> and for ec<strong>on</strong>omic evaluati<strong>on</strong> of investments in ERP/CRM<br />

systems c<strong>on</strong>ducted in Poland, as well as in other countries.<br />

Table 7 presents the percentage structure of resp<strong>on</strong>dents’ answers to questi<strong>on</strong>s c<strong>on</strong>cerning<br />

organisati<strong>on</strong>al problems related to a lack of ec<strong>on</strong>omic evaluati<strong>on</strong> in ERP/CRM IS projects’ realisati<strong>on</strong><br />

in the ex-ante and the ex-post perspective.<br />

Table 7: Organisati<strong>on</strong>al problems in ec<strong>on</strong>omic evaluati<strong>on</strong> of ERP/CRM IS projects’ investment in the<br />

ex-ante and the ex-post perspective<br />

Ex-ante Ex-post<br />

No. Problems % %<br />

1 Lack of knowledge<br />

about evaluati<strong>on</strong> 31% 9%<br />

2 Lack of interest 47% 55%<br />

3 Lack of data or informati<strong>on</strong> 15% 8%<br />

4 Lack of time 4% 16%<br />

5<br />

Source: Own study.<br />

Other 11% 14%<br />

The resp<strong>on</strong>dents could choose more than <strong>on</strong>e answer. 31% of resp<strong>on</strong>dents declared a lack of<br />

knowledge about ec<strong>on</strong>omic evaluati<strong>on</strong> in ERP/CRM IS project implementati<strong>on</strong> from the ex-ante<br />

perspective, while 9% of resp<strong>on</strong>dents declared a lack of corresp<strong>on</strong>ding knowledge from the ex-post<br />

perspective. It is worth noticing that 47% of resp<strong>on</strong>dents declared a lack of interest in the ec<strong>on</strong>omic<br />

330


Bartosz Wachnik<br />

evaluati<strong>on</strong> of an ERP/CRM system implementati<strong>on</strong> from the ex-ante perspective and 55% of<br />

resp<strong>on</strong>dents also declared a lack of interest in the ex-post perspective. Table 8 presents the<br />

percentage distributi<strong>on</strong> of answers to questi<strong>on</strong>s related to the types of difficulties identified in<br />

ec<strong>on</strong>omic evaluati<strong>on</strong> of ERP/CRM system investment projects. The resp<strong>on</strong>dents could choose more<br />

than <strong>on</strong>e answer. As part of the research, 62% of resp<strong>on</strong>dents indicated difficulties related to<br />

quantifying and indentifying benefits from an ERP/CRM informati<strong>on</strong> system investment and 46% of<br />

resp<strong>on</strong>dents indicated difficulties related to quantifying and indentifying expenditure resulting from an<br />

ERP/CRM informati<strong>on</strong> system investment.<br />

Table 8: Types of difficulties identified in ec<strong>on</strong>omic evaluati<strong>on</strong> of ERP/CRM system investment<br />

projects<br />

Types of difficulties<br />

%<br />

Difficulties related to quantifying and<br />

indentifying benefits from an ERP/CRM<br />

informati<strong>on</strong> system investment 62%<br />

Difficulties related to quantifying and<br />

indentifying expenditure resulting from an<br />

ERP/CRM informati<strong>on</strong> system investment 46%<br />

No problems encountered 6%<br />

Other 2%<br />

Source: Own study.<br />

4. C<strong>on</strong>clusi<strong>on</strong><br />

An ERP/CRM system implementati<strong>on</strong> project c<strong>on</strong>sists mostly of adapting a system to the client’s<br />

functi<strong>on</strong>al requirements through appropriate parameterising and programming works (Customisati<strong>on</strong>).<br />

Whether or not an ERP/CRM system can bring organisati<strong>on</strong>al improvement to a company depends<br />

both <strong>on</strong> a system’s functi<strong>on</strong>al possibilities and the inventi<strong>on</strong>, initiative and most importantly knowledge<br />

and experience of c<strong>on</strong>sultants who adapt it to the company’s needs.<br />

Summing up the results of research c<strong>on</strong>cerning choice strategy and ERP/CRM implementati<strong>on</strong>, we<br />

can notice that for the resp<strong>on</strong>dents the main reas<strong>on</strong> for choosing systems from this class is<br />

guaranteeing the company’s survival <strong>on</strong> the market while treating these systems as a platform for<br />

change and thus achieving saltatory innovati<strong>on</strong> after a completed implementati<strong>on</strong> is <strong>on</strong>ly in the<br />

sec<strong>on</strong>d and third positi<strong>on</strong> respectively.<br />

Observati<strong>on</strong> of the research results, c<strong>on</strong>cerning the criteria of decisi<strong>on</strong>-making in ERP/CRM class<br />

informati<strong>on</strong> system investment, leads us to the c<strong>on</strong>clusi<strong>on</strong> that in the choice of systems from this<br />

class, the majority of companies’ management do not link this investment to direct or indirect<br />

ec<strong>on</strong>omic profits for the company and focus mostly <strong>on</strong> the questi<strong>on</strong> related to the system<br />

guaranteeing the required functi<strong>on</strong>ality and <strong>on</strong> the cost of implementati<strong>on</strong> and usage.<br />

The main expectati<strong>on</strong>s of entrepreneurs in this class of systems is servicing basic requirements, i.e.<br />

correct servicing of sales processes, purchases, stock management, correct accounting, possibility of<br />

planning and manufacturing executi<strong>on</strong>, important marketing areas. In other words, ERP/CRM systems<br />

have to allow for performing basic and necessary activities that make current functi<strong>on</strong>ing of the<br />

company possible.<br />

Hence, we get a picture of the companies’ representatives’ sceptical and cautious approach to<br />

implementati<strong>on</strong> of ERP/CRM class informati<strong>on</strong> system as a source of immediate competitive edge<br />

and, c<strong>on</strong>sequently, achieving direct and indirect ec<strong>on</strong>omic profits. ERP and CRM class systems are<br />

l<strong>on</strong>g past the early phase of development, when they could be treated as a source of competitive<br />

edge. We have to admit that they are standardised soluti<strong>on</strong>s, functi<strong>on</strong>ally and technologically mature.<br />

Analysing appropriate group of ERP and CRM systems, i.e. dedicated to a selected group of clients,<br />

we can notice a big similarity when it comes to the variety of functi<strong>on</strong>alities and business soluti<strong>on</strong>s <strong>on</strong><br />

offer. ERP/CRM systems usually functi<strong>on</strong> <strong>on</strong> the basis of the same technological comp<strong>on</strong>ents, e.g.<br />

data bases. The product standard, or the products’ group standard, has already been devised and it is<br />

widely used. At the same time, during the last 10 years, the cost of investment in ERP/CR systems<br />

has noticeably decreased, the costs of ERP/CRM system licence have decreased by at least 20-30%,<br />

an hourly wages for c<strong>on</strong>sultants have stabilised in Poland <strong>on</strong> the level of 60 - 70 EUR.<br />

331


Bartosz Wachnik<br />

ERP and CRM systems have suddenly become an everyday good, just as access to mass media or<br />

means of transport. Entrepreneurs stopped treating informati<strong>on</strong> systems from this class as a source of<br />

temporary competitive edge, and a result a source of achieving direct and indirect ec<strong>on</strong>omic benefits,<br />

just as email access or having an eye-catching website is not treated as a source of competitive edge<br />

in companies. However, establishing and applying methods and tools allowing companies to perform<br />

ec<strong>on</strong>omic evaluati<strong>on</strong> of ERP and CRM class informati<strong>on</strong> systems in Poland and in other countries fall<br />

behind functi<strong>on</strong>al and technological development of this class of applicati<strong>on</strong>s.<br />

Summing up research results linked to ec<strong>on</strong>omic evaluati<strong>on</strong> of ERP/CRM class informati<strong>on</strong> systems<br />

implementati<strong>on</strong> investments; we can notice that resp<strong>on</strong>dents declare both from the ex-ante (63%) and<br />

the ex-post (72%) perspective that they have not performed an ec<strong>on</strong>omic analysis for investments in<br />

informati<strong>on</strong> systems from this class. Research c<strong>on</strong>cerning ec<strong>on</strong>omic evaluati<strong>on</strong> of IT investments<br />

c<strong>on</strong>ducted in 2002 in the USA showed that <strong>on</strong>ly 14% of resp<strong>on</strong>dents have not carried out an<br />

ec<strong>on</strong>omic evaluati<strong>on</strong> of the investment. Other research c<strong>on</strong>cerning ec<strong>on</strong>omic evaluati<strong>on</strong> of IT<br />

investments performed in 2002 am<strong>on</strong>gst US and selected Western <str<strong>on</strong>g>European</str<strong>on</strong>g> companies indicated<br />

that <strong>on</strong>ly 5% of resp<strong>on</strong>dents did not carry out an ec<strong>on</strong>omic evaluati<strong>on</strong> of the investment. Between<br />

2001 and 2006, P. Lech c<strong>on</strong>ducted a case analysis aimed at defining whether Polish companies,<br />

deciding to perform an IT project, i.e. company management support informati<strong>on</strong> systems<br />

implementati<strong>on</strong>, use a structured approach c<strong>on</strong>taining an element of ec<strong>on</strong>omic evaluati<strong>on</strong>. Interviews<br />

were carried out in 29 companies, of different size, planning to implement an integrated informati<strong>on</strong><br />

management system. According to P. Lech (Lech P., 2007), n<strong>on</strong>e of the organisati<strong>on</strong>s taking part in the<br />

research performed an ec<strong>on</strong>omic evaluati<strong>on</strong> of their planned investment other than comparing the<br />

cost of individual offers.<br />

Between 2004 and 2012, the author c<strong>on</strong>ducted a similar case analysis in order to define if Polish<br />

SMEs, deciding to implement a company management support informati<strong>on</strong> systems, use an approach<br />

including elements of a company’s ec<strong>on</strong>omic evaluati<strong>on</strong>. The analysis was carried out in a group of<br />

35 companies. According to the author, <strong>on</strong>ly <strong>on</strong>e company performed an ec<strong>on</strong>omic analysis of the<br />

planned project other than comparing the cost of individual offers. It is worth highlighting that Polish<br />

entrepreneurs, top and middle level managers are not highly motivated or interested in performing an<br />

ec<strong>on</strong>omic evaluati<strong>on</strong> of ERP/CRM class informati<strong>on</strong> systems implementati<strong>on</strong> projects before<br />

launching them (ex-ante). The main reas<strong>on</strong>s are:<br />

Lack of accessible, practical knowledge c<strong>on</strong>cerning the possibilities of using methods and<br />

techniques allowing to clearly and relatively easily carry out an ec<strong>on</strong>omic evaluati<strong>on</strong> of an<br />

investment into IS projects c<strong>on</strong>sisting of an ERP/CRM class systems implementati<strong>on</strong> from the exante<br />

perspective. Scientific centres, software producers and independent c<strong>on</strong>sulting companies<br />

do not provide sufficient services of ec<strong>on</strong>omic investments in IS projects. Research indicates that<br />

in their evaluati<strong>on</strong> company representatives rely mostly <strong>on</strong> quantitative and qualitative methods.<br />

Lack of awareness am<strong>on</strong>gst company representatives about the need to carry out an ec<strong>on</strong>omic<br />

evaluati<strong>on</strong> of an investment into IS projects c<strong>on</strong>sisting of an ERP/CRM class systems<br />

implementati<strong>on</strong> from the ex-ante perspective. We need to underline that ec<strong>on</strong>omic evaluati<strong>on</strong> of a<br />

planned ERP/CRM system investment should c<strong>on</strong>stitute an element of the project feasibility<br />

study, am<strong>on</strong>gst collected and defined functi<strong>on</strong>al and technological system requirements. Such<br />

feasibility studies should be performed before the selecti<strong>on</strong> of an informati<strong>on</strong> system and<br />

implementati<strong>on</strong> partner. As a result, companies do not c<strong>on</strong>duct project feasibility studies properly,<br />

which may result in choosing an inappropriate ERP/CRM class informati<strong>on</strong> system or the wr<strong>on</strong>g<br />

implementati<strong>on</strong> partner and have a negative impact <strong>on</strong> the project realisati<strong>on</strong>.<br />

A very important research result is the fact that 72% company representatives do not carry out an<br />

ec<strong>on</strong>omic evaluati<strong>on</strong> of their investment after completing an ERP/CRM informati<strong>on</strong> system project.<br />

According to the author, the same reas<strong>on</strong>s are valid in the ex-post perspective as in the ex-ante<br />

perspective. It is important to notice that 55% of resp<strong>on</strong>dents indicated a lack of interest in performing<br />

an analysis of the project ec<strong>on</strong>omic evaluati<strong>on</strong> ex-post, compared to 47% of resp<strong>on</strong>dents in the exante<br />

perspective. Performing an ec<strong>on</strong>omic evaluati<strong>on</strong> from the ex-post perspective may reveal<br />

mistakes in the selecti<strong>on</strong> process of an ERP/CRM system or implementati<strong>on</strong> partner and, additi<strong>on</strong>ally,<br />

allow for verifying organisati<strong>on</strong>al mistakes in the process of implementing informati<strong>on</strong> systems.<br />

Hence, the lower number of resp<strong>on</strong>dents interested in c<strong>on</strong>ducting an ec<strong>on</strong>omic evaluati<strong>on</strong>.<br />

Research shows that the resp<strong>on</strong>dents indicated two equivalent main difficulties in performing an<br />

ec<strong>on</strong>omic evaluati<strong>on</strong>, i.e.<br />

332


Bartosz Wachnik<br />

62% of resp<strong>on</strong>dents indicated difficulties related to quantifying and identifying the benefits<br />

resulting from investing in an ERP/CRM informati<strong>on</strong> system. In many cases it was impossible to<br />

clearly establish which benefits of an ERP system implementati<strong>on</strong> result from its implementati<strong>on</strong>,<br />

and which from the company’s organisati<strong>on</strong>al and technological activity. We can look at the<br />

example of design and use of a product price-list structure including complex price discounts that<br />

was implemented in a distributi<strong>on</strong> company. After implementing the price-list structure<br />

functi<strong>on</strong>ality in the ERP system, the company noticed an increase of a few percent in the return<br />

<strong>on</strong> sales. It is very difficult to definitely determine the source of quantified benefit - whether it was<br />

the implementati<strong>on</strong> of a process innovati<strong>on</strong>: designing the price-list, or its implementati<strong>on</strong> in the<br />

sales management module of the ERP system. Very frequently, after the implementati<strong>on</strong>, an ERP<br />

system becomes the source of reliable informati<strong>on</strong> generated in a shorter period of time and more<br />

easily than before the implementati<strong>on</strong>, e.g. sending text messages or creating attractive graphs.<br />

Nevertheless, if the informati<strong>on</strong> recipients are not sufficiently prepared to use it and miss the<br />

chance, the company will not achieve an ec<strong>on</strong>omic effect. Also in this case, it is difficult to clearly<br />

identify benefits resulting from the system implementati<strong>on</strong>.<br />

46% of resp<strong>on</strong>dents declared problems linked to quantifying and identifying the cost resulting<br />

from an ERP/CRM informati<strong>on</strong> system investment. Investing in an ERP system purchase entails<br />

both expenditure <strong>on</strong> equipment purchase and software licence at the beginning of a project and<br />

covering costs of system maintenance and its development for 4-6 years. Investment expenditure<br />

is linked to spending financial funds and internal expenditure <strong>on</strong> tasks performed by internal<br />

specialists. In many cases, it is not possible to definitely determine which expenditure over the<br />

years – or which part of the expenditure – is c<strong>on</strong>nected with an ERP/CRM system implementati<strong>on</strong><br />

project.<br />

Comparative research shows that the identified difficulties c<strong>on</strong>cerning quantifying benefits and costs<br />

have also been indicated by other research (Ballatine J.A., Galliers R.D., Stray S.J., 1999). It is worth<br />

highlighting that more resp<strong>on</strong>dents indicated identifying and quantifying benefits as a difficulty in<br />

carrying out an ERP/CRM system investment evaluati<strong>on</strong>. To sum up, the observati<strong>on</strong> of research<br />

results leads to the following c<strong>on</strong>clusi<strong>on</strong>s. Am<strong>on</strong>gst the entrepreneurs and top management of a big<br />

group of Polish companies, ERP/CRM systems are not treated as avant-garde IT soluti<strong>on</strong>s that can<br />

give a temporary competitive edge in the mid-term, and, c<strong>on</strong>sequently, a direct or indirect ec<strong>on</strong>omic<br />

benefit. However, informati<strong>on</strong> systems from this class c<strong>on</strong>stitute necessary IT soluti<strong>on</strong>s, indispensable<br />

for c<strong>on</strong>ducting business activity, like email access, electricity or even water. In most cases,<br />

entrepreneurs and management do not carry out an ec<strong>on</strong>omic analysis while performing ERP/CRM<br />

system investments, neither in the ex-ante nor the ex-post perspective. According to the author, there<br />

are two main reas<strong>on</strong>s for this phenomen<strong>on</strong>. First of all, there is a lack of verified methods allowing this<br />

kind of ec<strong>on</strong>omic analysis that could be adapted for a specific type of ERP/CRM system. Sec<strong>on</strong>dly,<br />

entrepreneurs and company managers lack determinati<strong>on</strong> to solve the problem of ec<strong>on</strong>omic<br />

evaluati<strong>on</strong> of management support informati<strong>on</strong> systems.<br />

References<br />

Ballatine J.A.,Galliers R.D.,Stray S.J, “Informati<strong>on</strong> Systems/Technology Evaluati<strong>on</strong> Practices: Evidence from UK<br />

Organisati<strong>on</strong>s”, in: L. Willcocks, S. Lester (eds.), Bey<strong>on</strong>d the IT Productivity Paradox: Assessment Issues,<br />

John Wiley&S<strong>on</strong>s, Chichester 1999.<br />

Cr<strong>on</strong>holm S., Goldkuhl G., “Strategies for Informati<strong>on</strong> System Evaluati<strong>on</strong> – Six Generic Types”,<br />

http://www.ejise.com, EJISE 2003.<br />

Cypryjański J., The Basic Methods of IS Investment Ec<strong>on</strong>omic Effectiveness Evaluati<strong>on</strong> in Enterprises, University<br />

of Szczecin, Szczecin 2007<br />

Dudycz H., Dyczkowski M., Effectiveness of IS Investments. The Basic Evaluati<strong>on</strong> Methods and Usage<br />

Examples, Publishing house of Wrocław University, Wrocław 2006.<br />

Lech P., Methodics of MIS Investment Ec<strong>on</strong>omic Evaluati<strong>on</strong>, Publishing house of Gdańsk University, Gdańsk<br />

2007.<br />

Niedźwiedziński M., Evaluati<strong>on</strong> of an Enterprise’s IS Projects, Acta Universitatis Lodziensis, Publishing house of<br />

Łódź University, Łódź 1989<br />

Powell P.L., “Evaluati<strong>on</strong> of informati<strong>on</strong> technology investments: Business as usual?” [in] L. P. Willcocks & S.<br />

Lester (eds.), Bey<strong>on</strong>d the IT Productivity Paradox, John Wiley & S<strong>on</strong>s, Chichester 1999, page 151.<br />

Paul L.G., Tate P., CFO Mind Shift: Technology Creates Value, CFO Publishing Corporati<strong>on</strong>, Bost<strong>on</strong>, MA 2002.<br />

Remenyi D.,M<strong>on</strong>ey A.,Sherwood-Smith M., Effective Measurement and <strong>Management</strong> of IT Costs & Benefits,<br />

Butterworth-Heinemann, Woburn, Massachusetts 2000.<br />

Weill P., Broadbent M., Leveraging the New Infrastructure: How Market Leaders Capitalize <strong>on</strong> Informati<strong>on</strong><br />

Technology, Harvard Business School Press, Bost<strong>on</strong>, Massachusetts 1998<br />

333


Towards an Understanding of Enterprise Architecture<br />

Analysis Activities<br />

Haining Wan 1, 2 and Sven Carlss<strong>on</strong> 2<br />

1<br />

Key Lab of Informati<strong>on</strong> Systems Engineering, School of Informati<strong>on</strong> Systems<br />

and <strong>Management</strong>, Nati<strong>on</strong>al University of Defense Technology, Changsha,<br />

P.R.China<br />

2<br />

Department of Informatics, Lund University, Lund, Sweden<br />

Haining.Wan@ics.lu.se<br />

Sven.Carlss<strong>on</strong>@ics.lu.se<br />

Abstract: The c<strong>on</strong>notati<strong>on</strong> of the term of Enterprise Architecture (EA) analysis varies from c<strong>on</strong>text to c<strong>on</strong>text.<br />

Aiming at, in part, promoting the understanding and reducing the possible misc<strong>on</strong>cepti<strong>on</strong> of EA analysis, in order<br />

to characterize, classify, and distinguish the c<strong>on</strong>notati<strong>on</strong>s of EA analysis, six interrelated types of activities are<br />

identified: (I) System thinking; (II) Modeling; (III) Measuring; (IV) Satisfying; (V) Comparing with requirements,<br />

and (VI) Comparing alternatives. The paper starts with the EA lifecycle management, and then addresses the<br />

main tasks in different stages in the EA lifecycle process. After that, the meaning of each type of activities and<br />

their interrelati<strong>on</strong>ships are discussed. The usage of the six types of activities is illustrated through several<br />

scenarios of EA analysis. The main c<strong>on</strong>tributi<strong>on</strong> of the paper is twofold: first, it articulates that there is a broader<br />

variety of understandings of the term EA analysis than we could imagine; sec<strong>on</strong>d, it provides the possibility and<br />

feasibility for researchers to explain and customize, i.e. for authors to clarify and for audiences to grasp, the<br />

meaning of EA analysis using certain combinati<strong>on</strong>s of the six activities.<br />

Keywords: enterprise architecture analysis, enterprise architecture, analysis and design, evaluati<strong>on</strong> and<br />

assessment, validati<strong>on</strong> and verificati<strong>on</strong>, enterprise analysis<br />

1. Introducti<strong>on</strong><br />

Enterprise Architecture (EA) is of great importance for aligning enterprise IT-assets with enterprise<br />

business and strategy. EA is a state-of-the-art alternative for achieving enterprise management goals<br />

such as improving business performance, decreasing resource use, c<strong>on</strong>trolling risk and complexity,<br />

and coping with uncertain envir<strong>on</strong>ment. Through the planning of transiti<strong>on</strong> from the baseline<br />

architecture to the target <strong>on</strong>e and finally, gearing the IT assets to the needs of enterprise business,<br />

the c<strong>on</strong>cept of EA is receiving more and more attenti<strong>on</strong>s both from academy and industry.<br />

However, “having an enterprise architect does not guarantee a successful EA practice” (Sessi<strong>on</strong>s<br />

2007), neither necessarily does having an EA framework. As to facilitating a success, EA analysis is<br />

important and indispensable for two reas<strong>on</strong>s:<br />

It supports organizati<strong>on</strong>s to get a better understanding of EA (both real and designed) itself. For<br />

instance, in EA initiatives, while artifacts, e.g. models, are built with layers and views (or<br />

viewpoints), enterprise data is collected, and then, analysis is intended to find gaps, weaknesses,<br />

pitfalls and shortfalls that exist in the enterprise.<br />

It supports organizati<strong>on</strong>s in their EA decisi<strong>on</strong>-making. EA transiti<strong>on</strong> is often very expensive and<br />

highly risky; furthermore, with respects to complexity, it is vital to do in-depth analysis before<br />

making decisi<strong>on</strong>s.<br />

This paper is intended to react to such a reality: (1) C<strong>on</strong>cerning the term EA analysis, there is no<br />

formal definiti<strong>on</strong> which is widely accepted or in comm<strong>on</strong> use. (2) The c<strong>on</strong>notati<strong>on</strong>s of EA analysis vary<br />

a lot in literature. The c<strong>on</strong>tents and boundaries of the term are left vague and ambiguous. (3) Little of<br />

the literature with the keyword EA analysis makes the c<strong>on</strong>notati<strong>on</strong> of EA analysis explicit enough. (4)<br />

Few authors of the literature would think that it is important to clarify the meaning of EA analysis. They<br />

just take it for granted. Of course, it is out of the scope of the paper to develop a <strong>on</strong>e-size-fit-all<br />

definiti<strong>on</strong> of EA analysis. Instead, in this work, the possibility and feasibility to develop a better<br />

understanding towards EA analysis is provided.<br />

The remainder of the paper is structured as follows. Secti<strong>on</strong> 3 proposes a framework of the six core<br />

types of activities and the relati<strong>on</strong>ships between activities after discussing EA and EA analysis in<br />

Secti<strong>on</strong> 2. Secti<strong>on</strong> 4 presents nine typical scenarios for illustrating how the framework with its six<br />

types of activities can be used. Secti<strong>on</strong> of C<strong>on</strong>clusi<strong>on</strong>s finalizes the paper.<br />

334


2. EA and EA analysis<br />

Haining Wan and Sven Carlss<strong>on</strong><br />

Enterprise software-intensive systems, e.g., Enterprise Systems (ES) and Enterprise Resource<br />

Planning (ERP) systems, are becoming increasingly complex and expensive and at the same time<br />

there is a need for flexibility and agility since organizati<strong>on</strong>s’ envir<strong>on</strong>ments are becoming more<br />

turbulent and high-velocity. In such a c<strong>on</strong>text, it is vital for an organizati<strong>on</strong> to keep its investment at a<br />

reas<strong>on</strong>able level, i.e. to try to make a good enough alignment between IT (including IT investment)<br />

and the organizati<strong>on</strong>’s strategy, business model, and business processes.<br />

EA has been emerging as an interdisciplinary subject since Zachman’s Framework for informati<strong>on</strong><br />

system architecture was presented (1987). It should be noticed that not <strong>on</strong>ly IT-related informati<strong>on</strong> is<br />

included in EA research and practice, but business-related issues, such as processes, organizati<strong>on</strong>,<br />

strategy, capacity, and so <strong>on</strong>, should also be included in EA initiatives(Mayo & Tiemann 2005; Meyers<br />

2011).<br />

Normally, enterprises employ EA Frameworks in initiatives to accomplish their alignment. There are<br />

many EA Frameworks linked: Zackman Frameworks (Zachman, John Alexander 2001; 2004, 2011),<br />

TOGAFs (OpenGroup 2003, 2010), DODAFs (DoD Architecture Framework Working Group 2003,<br />

2009), FEAF (The Chief Informati<strong>on</strong> Officers Council 1999), TEAF (US Department of the Treasury<br />

Chief Informati<strong>on</strong> Officer Council 2000) and some other frameworks bel<strong>on</strong>ging to some particular<br />

enterprises. Based <strong>on</strong> different experiences and knowledge, many organizati<strong>on</strong>s and researchers had<br />

proposed EA frameworks of their own with definiti<strong>on</strong>s of EA.<br />

Practically, in pursuit of success, c<strong>on</strong>tinuous EA management is the choice of factualisms, i.e. the life<br />

cycle management. There are many similar expressi<strong>on</strong>s <strong>on</strong> the architecture lifecycle process, e.g.<br />

TOGAF ADM(OpenGroup 2010), PDCA(Plan Do Check Act) (DoD Architecture Framework<br />

Working Group 2009; Moen & Norman 2006) and Architecture process (Create EA Apply and Use<br />

EA Maintaining EA results), etc.<br />

The initial objective of an EA analysis is to understand the enterprise’s strategy and develop soluti<strong>on</strong>s<br />

for the gap between baseline and desired target. EA is oftentimes viewed as a tool to provide<br />

evidences for enterprise-wide decisi<strong>on</strong>-making, similarly, EA analysis could also be regarded as a<br />

way to provide evidences for architecture process management, and to support the main tasks<br />

depicted in Table 1. To be c<strong>on</strong>cise, PDCA process is directly used in the table. It should be noted that<br />

the tasks in steps within the PDCA process help to shape Secti<strong>on</strong>3 and Secti<strong>on</strong>4.<br />

Table 1: PDCA process with the main tasks<br />

Phases Main tasks to do<br />

Plan to make clear the intent, the requirements, the visi<strong>on</strong>, the strategy, the target<br />

Do to collect informati<strong>on</strong> and data, to build up models, to quantify EA<br />

to check the process of creati<strong>on</strong>, to make sure both the procedure and the result are right, to<br />

Check<br />

check the effectiveness and efficiency of the architecture artifacts<br />

to apply the result of artifacts into the real enterprise, to implement the architecture artifacts, to<br />

Acti<strong>on</strong> enable the transiti<strong>on</strong> and migrati<strong>on</strong> of the enterprise architecture, to maintain and govern the<br />

architecture in order to keep the enterprise architecture in good c<strong>on</strong>diti<strong>on</strong>, to launch a new<br />

cycle if needed<br />

3. The core types of activities in EA analysis<br />

Based <strong>on</strong> literature review and narrative, this part of the paper will discuss the many core types of<br />

activities in EA analysis. Of course, another source is my pers<strong>on</strong>al understanding of EA analysis.<br />

3.1 System thinking<br />

System thinking is defined as “an epistemology which, when applied to human activity is based up<strong>on</strong><br />

the four basic ideas: emergence, hierarchy, communicati<strong>on</strong>, and c<strong>on</strong>trol as characteristics of systems.<br />

When applied to natural or designed systems the crucial characteristics is the emergent properties of<br />

the whole.” (Checkland 1981, p. 381) and “the art and science of making reliable inferences about<br />

behavior by developing an increasingly deep understanding of underlying structure” (Richm<strong>on</strong>d 1987).<br />

However, system thinking here represents problem-solving social human activity embodied in the<br />

‘epistemology, art and science’. System thinking is included as an important element of ’system<br />

335


Haining Wan and Sven Carlss<strong>on</strong><br />

engineering’ and ‘system analysis’ in the methodological cycle towards problem-solving problems<br />

(Checkland 1981, pp. 147, 54).<br />

In the PCCAM process, the enterprise itself is viewed as a dynamic and complex whole system.<br />

Different parts of the system within the enterprise are interacting with each other as a structured<br />

functi<strong>on</strong>al unit. It depends up<strong>on</strong> the c<strong>on</strong>cern, interests of system thinker, and the granularity to<br />

determine the ingredients of the collecti<strong>on</strong> of system parts when there is a problem to be solved.<br />

System thinking for EA analysis has two characteristics. First, holistic c<strong>on</strong>siderati<strong>on</strong> of both enterprise<br />

business-related and IT-related issues. System thinking in problem-solving of EA analysis takes a<br />

holistic look at enterprise. With system thinking, analysts justify what the architecture within an<br />

organizati<strong>on</strong> is (as-is) and what it ought to be (to-be). Sec<strong>on</strong>d, separati<strong>on</strong> of c<strong>on</strong>cern. Analysts should<br />

dig in-depth with views (viewpoints) to focus <strong>on</strong> some particular parts of the enterprise and get detail<br />

models according the c<strong>on</strong>cerns. With the aid of system thinking, analysts divide the enterprise into<br />

parts and make classificati<strong>on</strong> of layers with some special views or viewpoints. In such a (thought)<br />

process, views are the result of what could be seen, the viewpoints are the points from which to get<br />

the views (Maier, Emery & Hilliard 2004).<br />

System thinking is applicable in all the stages of the EA process, no matter whether the existence<br />

form is visible or invisible, tangible or intangible, c<strong>on</strong>scious or unc<strong>on</strong>scious, direct or indirect, implicit<br />

or explicit, and formal or informal. System thinking is of great significance to deal with complex and<br />

unstructured problem existing within the domain of EA.<br />

3.2 Modeling<br />

EA Modeling is intended to build up models and to collect informati<strong>on</strong> and data for describing or<br />

prescribing EA. This type of EA activities aims to provide stakeholders with comm<strong>on</strong> artifacts (models)<br />

for communicati<strong>on</strong>s.<br />

With EA Modeling from the real world to logical world, an integrated whole enterprise could be divided<br />

into parts according to the c<strong>on</strong>cern, framework, and viewpoint of all types of stakeholders. And finally,<br />

there are EA models with more details in the logical (artificial) world, either descriptive or prescriptive.<br />

Collecti<strong>on</strong>s of EA artifacts could also be called layers, slices or views, details and models, shown as in<br />

Figure 1.<br />

Figure 1: Analysis and synthesis of EA with modeling<br />

Actually, EA modeling is a sort of work c<strong>on</strong>sisting of:<br />

Design. As a work of design, modelers usually rely heavily <strong>on</strong> imaginati<strong>on</strong>, their experience and<br />

knowledge. It is a work of innovati<strong>on</strong>.<br />

Decisi<strong>on</strong>-making. To some extern, EA modeling is an attempt to reduce uncertainty. From a topdown<br />

point of view, the modelers’ endeavor to make clear the details in the models. Modelers<br />

have to make choices, adopt some design, and aband<strong>on</strong> others.<br />

Analogy and metaphor (Khoury & Simoff 2004; McFadden 2001). In the mapping from the real<br />

world to the logical world, <strong>on</strong>e side is the enterprise and envir<strong>on</strong>ment; the other side is the<br />

artifacts (models or data, descripti<strong>on</strong>s or prescripti<strong>on</strong>s). With the aid of analogy and metaphor,<br />

analysts “rebuild” from the real world to the logical world in terms of comp<strong>on</strong>ents and the<br />

relati<strong>on</strong>ships between comp<strong>on</strong>ents by symbols and modeling language.<br />

Informati<strong>on</strong> reducti<strong>on</strong> and informati<strong>on</strong> encapsulati<strong>on</strong>. As the communicati<strong>on</strong> medium am<strong>on</strong>g<br />

stakeholders and the blueprint of transiti<strong>on</strong> plan, the less complicated EA models are better<br />

models for stakeholders(Sessi<strong>on</strong>s 2007), especially executives. Every model focuses <strong>on</strong> <strong>on</strong>ly<br />

limited aspects. Modelers have to exclude irrelevant informati<strong>on</strong> in every special model so as to<br />

gain more c<strong>on</strong>cise <strong>on</strong>es.<br />

336


3.3 Measuring<br />

Haining Wan and Sven Carlss<strong>on</strong><br />

EA is complicated and EA initiatives are often risky, expensive and can have a high degree of<br />

uncertainty. Enterprise transformati<strong>on</strong>s and EA migrati<strong>on</strong>s c<strong>on</strong>note change, and change brings about<br />

new assignments of duties and rights. Probably, this means that there would be uncomfortableness<br />

and even resistances. The executives need to overcome the resistances from both internal and<br />

external stakeholders. As a result, enterprise transformati<strong>on</strong> and EA migrati<strong>on</strong> are often associated<br />

with painful things. For this reas<strong>on</strong>, analysts try to guarantee the success rate. Measuring is a type of<br />

underpinning and underlying activity to facilitate the success of EA practice. More basically, EA could<br />

not be readily managed and c<strong>on</strong>trolled in practice if it is not measured.<br />

Artifacts are tools and mediums which analysts use to understand the performance of the real<br />

enterprise. The artifacts, normally, are compared with the requirement of the enterprise before final<br />

EA acti<strong>on</strong>s, e.g. decisi<strong>on</strong>-making, implementati<strong>on</strong>.<br />

Both qualitative and quantitative methods could be used in measurement. C<strong>on</strong>cerning measurement<br />

for analyzing the EA, aspects could be divided into two parts, shown as in Table 2. The table is based<br />

<strong>on</strong> pers<strong>on</strong>al experience and understanding.<br />

Table 2: Two parts of EA measurement<br />

Mapping of EA<br />

EA itself(both artifacts and real world)<br />

(from real world to models)<br />

Functi<strong>on</strong>al and n<strong>on</strong>-functi<strong>on</strong>al performance The rightness of both artifacts and the<br />

Object<br />

Quality of<br />

what<br />

Focus<br />

Example of<br />

activity<br />

according to the spatial-temporal attributes<br />

Quality of the EA itself (reflecting the<br />

goodness of design and real enterprise)<br />

Comp<strong>on</strong>ents and their relati<strong>on</strong>ships in the<br />

same EA<br />

Measuring for evaluati<strong>on</strong> and assessment,<br />

e.g. measuring performance based <strong>on</strong> model<br />

simulati<strong>on</strong> and executi<strong>on</strong><br />

process to build up the artifacts<br />

Quality of the models (reflecting the<br />

goodness of mapping and visualizing)<br />

Both the models and the real enterprise<br />

system<br />

Measuring for validati<strong>on</strong> and verificati<strong>on</strong>,<br />

e.g. c<strong>on</strong>sistency checking<br />

3.4 Satisfying<br />

The thoughts to get a good enough EA by designing <strong>on</strong>ly <strong>on</strong>ce seems not to be practical. The<br />

classical pattern to make a success by <strong>on</strong>ly designing <strong>on</strong>ce does not work in the domain of EA design<br />

(Webnova Inc. 2009). On the c<strong>on</strong>trary, analysts have to improve their design step by step, again and<br />

again, till there is a satisfactory design. Satisfactory means that stakeholder’ requirements are met, or<br />

there is a compromise between stakeholders to reach a c<strong>on</strong>sensus.<br />

Zia et al. (2011) proposed to view EA analysis as a MCDM (multiple criteria decisi<strong>on</strong> making)<br />

problem. That is to select correct architecture from designs, and view such a selecti<strong>on</strong> problem as a<br />

MCDM problem. Two difficulties exist: <strong>on</strong>e is to transform unstructured problem of EA analysis into<br />

structured problem of MCDM by c<strong>on</strong>ceptualizing the problem with c<strong>on</strong>crete objective functi<strong>on</strong>s and<br />

c<strong>on</strong>straint c<strong>on</strong>diti<strong>on</strong>s, and the other is to transform the MCDM problem into SCDM (single criteria<br />

decisi<strong>on</strong> making) problem by c<strong>on</strong>verting between objective functi<strong>on</strong>s and c<strong>on</strong>straint c<strong>on</strong>diti<strong>on</strong>s.<br />

Note that, EA analysis problems are often open <strong>on</strong>es with open objective functi<strong>on</strong>s and c<strong>on</strong>straints<br />

rather than closed <strong>on</strong>es. Those who want to do optimizati<strong>on</strong> should first transform real open problems<br />

into closed <strong>on</strong>es. However, <strong>on</strong>e of the most important characteristics of such a problem<br />

transformati<strong>on</strong> is not whether the artifacts are right or wr<strong>on</strong>g; yet, it is whether or not it works well<br />

enough.<br />

Practically, it is often impossible for analysts to find the optimal soluti<strong>on</strong>(s) of the EA optimizati<strong>on</strong><br />

problem, sometimes the sub-optimal soluti<strong>on</strong>(s) and even the satisfactory soluti<strong>on</strong> could be the final<br />

choice.<br />

For a better result of EA optimizati<strong>on</strong> problem, three sub-types of activities could be d<strong>on</strong>e:<br />

Sensitive analysis. It is to study how the inputs uncertainty of EA analysis will influence the<br />

outputs result. There are two approaches. One to change the value of system attributes, and the<br />

other to change the set of system attributes tree.<br />

337


Haining Wan and Sven Carlss<strong>on</strong><br />

Trade-off analysis. While the sensitive analysis is d<strong>on</strong>e with the changes of system attributes,<br />

trade-off analysis could be d<strong>on</strong>e by altering the enterprise model. EA decisi<strong>on</strong>s are full of tradeoffs,<br />

between l<strong>on</strong>g-term and short-term (referring to transiti<strong>on</strong> plan), between different<br />

organizati<strong>on</strong> deployments (referring to department deployment plan), between different c<strong>on</strong>tents<br />

of organizati<strong>on</strong> deployment (referring to system suit).<br />

Soluti<strong>on</strong>s formulati<strong>on</strong> and revisi<strong>on</strong>. Associated with satisfying, soluti<strong>on</strong>s for enterprise<br />

misalignment and change management are often formulated. Meanwhile, for the reas<strong>on</strong> that the<br />

collecti<strong>on</strong> of system attributes are multidimensi<strong>on</strong>al, the stakeholders are also multiple, soluti<strong>on</strong>s<br />

often need revisi<strong>on</strong>s before agreements and c<strong>on</strong>sensuses could be reached.<br />

3.5 Comparing with requirements<br />

To build a model without purposes and requirements does not make sense. It is vital to identify the<br />

requirements in that different stakeholders hold different c<strong>on</strong>cerns, and in different c<strong>on</strong>texts, EA<br />

project is intended to deal with different kinds and aspects of issues.<br />

The requirements c<strong>on</strong>sist of two parts:<br />

The requirement of enterprise architecture. This kind of requirement is about the functi<strong>on</strong>al and<br />

n<strong>on</strong>-functi<strong>on</strong>al setting of enterprise architecture (problem of what and how to do EA analysis,<br />

syntax and semantics). It comes from the chain: EA visi<strong>on</strong> strategy goal requirement.<br />

This requirement shapes the directi<strong>on</strong> of the EA, and it could be embodied within enterprise<br />

visi<strong>on</strong>, the strategy, the gap between the ideal EA status (As Is) and the reality (To Be).<br />

The requirement for what to do EA analysis. It is about the c<strong>on</strong>text in which EA analysis is needed<br />

(problem of why, pragmatics). Sometimes, this requirement has greater influence <strong>on</strong> choosing the<br />

method and approach to do EA analysis.<br />

3.6 Comparing alternatives<br />

As menti<strong>on</strong>ed above, usually there is no optimized soluti<strong>on</strong>, but <strong>on</strong>ly satisfactory <strong>on</strong>e. The<br />

performances of some soluti<strong>on</strong>s in some particular dimensi<strong>on</strong>s are better than others. Comparing<br />

alternatives is to observe the performance differences of alternatives in all kinds of dimensi<strong>on</strong>s. Based<br />

<strong>on</strong> the differences, advices for making choice begin to take shape.<br />

In EA initiatives, choosing the right <strong>on</strong>e from alternatives is an important thing. Analysts could obtain a<br />

whole appraisement about the EA through modeling, measurement, and satisfying. According to the<br />

appraisement, analysts could make comparis<strong>on</strong> and sort them in order, and sometimes they apply<br />

sensitive analysis again so as to facilitate observing the possible change of the appraisement. This is<br />

very comm<strong>on</strong> in the c<strong>on</strong>text of selecting from several EA soluti<strong>on</strong> candidates.<br />

3.7 Relati<strong>on</strong>ships between types of activities<br />

The interrelati<strong>on</strong>ships between the six types of activities are depicted in Figure 2. The six activities<br />

could be divided into three categories or levels, i.e. fundamental level including system thinking and<br />

comparing of requirement; main level including modeling, measuring, and satisfying; and decisi<strong>on</strong>oriented<br />

level including comparing between alternatives. Fundamental implies that it is basic,<br />

elementary and primary. Main means principal, mainstream and comm<strong>on</strong>. As the name suggests, the<br />

top level, i.e. decisi<strong>on</strong>-oriented activity is ultimately and directly useful for decisi<strong>on</strong> making.<br />

The directi<strong>on</strong>s of arrows indicate the supporting directi<strong>on</strong>s. Obviously, there are two implicati<strong>on</strong>s for<br />

the supporting relati<strong>on</strong>ships, i.e. the supporting relati<strong>on</strong>s in the same level and that crossing levels.<br />

Another point important in the interrelati<strong>on</strong>ships is that two threads could be identified, artifacts and<br />

architecture evoluti<strong>on</strong>. All the six types of activities are closely related with EA artifacts (e.g.<br />

models). The crafting and using of artifacts witness the informati<strong>on</strong> flows (inputs and outputs) between<br />

activities and supporting directi<strong>on</strong>s. EA has an artificial life, and architecture evoluti<strong>on</strong> implies that<br />

there is a temporal sequence for EA programs and activities in EA programs. This also exposes the<br />

relati<strong>on</strong>ships of supporting in the figure, specifically, the temporal sequence of c<strong>on</strong>ducting the<br />

activities in order to form a c<strong>on</strong>tinuous EA management.<br />

338


Haining Wan and Sven Carlss<strong>on</strong><br />

Figure 2: Interrelati<strong>on</strong>ships of six activities<br />

Razavi et al. (2011) introduced AHP (Analytic Hierarchy Process) approach to do enterprise<br />

architecture analysis. This approach could be used as a c<strong>on</strong>crete example to explain the<br />

interrelati<strong>on</strong>ships. First, System thinking is needed in each step in the whole process to reduce<br />

mistakes and possible c<strong>on</strong>tradicti<strong>on</strong>s, to improve the analysis results. Sec<strong>on</strong>d, the enterprise is<br />

modeled as a hierarchy according to the system attributes. Third, based <strong>on</strong> the results of modeling,<br />

measurement is d<strong>on</strong>e to calculate the performances layer after layer. Fourth, based <strong>on</strong> the results of<br />

modeling and measuring, sensitive study and trade-off analysis is c<strong>on</strong>ducted through the modificati<strong>on</strong><br />

of both system attributes (and sub-attributes) and the value of system attributes (and sub-attributes).<br />

Fifth, the analysts need to bear in mind the requirement in the whole EA analysis process. Sixth,<br />

based <strong>on</strong> the results of modeling, measuring and satisfying, the performances of different alternatives<br />

could be compared, and finally, decisi<strong>on</strong> is made. According to the six activities of EA analysis,<br />

potential c<strong>on</strong>notati<strong>on</strong>s of the term EA analysis are illustrated in Table 3, with each c<strong>on</strong>notati<strong>on</strong><br />

differing from the other two c<strong>on</strong>notati<strong>on</strong>s. With the requirements in hand and the questi<strong>on</strong> ‘whether or<br />

not and to which extent these types of activities are involved’, the c<strong>on</strong>notati<strong>on</strong>s seem to be capable of<br />

being portrayed.<br />

Table 3: Possible c<strong>on</strong>notati<strong>on</strong>s of EA analysis<br />

Potential C<strong>on</strong>notati<strong>on</strong>s Remarks<br />

Documentati<strong>on</strong>,<br />

communicati<strong>on</strong>, presentati<strong>on</strong>,<br />

descripti<strong>on</strong>s of enterprise and<br />

the enterprise architecture,<br />

etc.<br />

Literature examples: Buckl<br />

(2011), de Boer et al. (2005).<br />

Measurement, evaluati<strong>on</strong>,<br />

assessment, validati<strong>on</strong> and<br />

verificati<strong>on</strong>, etc.<br />

Literature examples: Johns<strong>on</strong><br />

et al. (2007), Lagerstrom et al.<br />

(2010).<br />

Soluti<strong>on</strong>s formulati<strong>on</strong> and<br />

decisi<strong>on</strong> support, etc.<br />

Literature examples: Razavi<br />

(2011), Zia et al. (2011).<br />

4. Illustrati<strong>on</strong>s and some typical scenarios<br />

There are several meanings, e.g., using structured<br />

analysis and object-oriented analysis method to get<br />

a better understanding of the enterprise. It may focus<br />

<strong>on</strong> the procedures to build artifacts, and oftentimes,<br />

some modeling languages, modeling tools, and<br />

database are introduced. With analysis, artifacts are<br />

created, and used to express the design thoughts.<br />

Quality and quantity of EA become the focus, and<br />

usually, criteria tree, variables, system properties,<br />

metrics and KPIs (Key Performance Indicators) are<br />

introduced in order to measure, evaluate, assess,<br />

validate, and verify EA.<br />

The main ends are to formulate or introduce some<br />

enterprise soluti<strong>on</strong>s, and then do some analysis <strong>on</strong><br />

the soluti<strong>on</strong>s, including revisi<strong>on</strong>, optimizati<strong>on</strong> and<br />

comparis<strong>on</strong>. Also typically it emphasizes assisting<br />

executives making decisi<strong>on</strong>s about the soluti<strong>on</strong>s.<br />

The ultimate soluti<strong>on</strong> could either be <strong>on</strong>e of the<br />

opti<strong>on</strong>s with revisi<strong>on</strong>, or a combinati<strong>on</strong> of some<br />

particular opti<strong>on</strong>s with trade-offs.<br />

Corresp<strong>on</strong>ding<br />

Combinati<strong>on</strong> of<br />

activities<br />

system thinking,<br />

modelling,<br />

comparing with<br />

requirements.<br />

system thinking,<br />

modelling,<br />

measuring,<br />

comparing with<br />

requirements, and<br />

sometimes<br />

satisfying<br />

All the six<br />

activities included<br />

In order to illustrate how the six types of activities can be applied in EA practice, this secti<strong>on</strong> will first<br />

introduce several classic scenarios that could be distilled of EA analysis form the EA lifecycle process.<br />

339


Haining Wan and Sven Carlss<strong>on</strong><br />

From the EA lifecycle process of view, there are many scenarios that could be encountered frequently<br />

in EA practice. With the main works to do in Table 1, the scenarios could be sorted as below:<br />

S1: To make EA descripti<strong>on</strong>s.<br />

S2: To form EA prescripti<strong>on</strong>s or soluti<strong>on</strong> alternatives.<br />

S3: To make assessments while designing or implementing enterprise architecture.<br />

S4: To make tests and measurements <strong>on</strong> how well the design performs.<br />

S5: To make an investigati<strong>on</strong> in order to improve the EA.<br />

S6: To check whether the model is built right (i.e. verificati<strong>on</strong>).<br />

S7: To check whether the model is the right <strong>on</strong>e (i.e. validati<strong>on</strong>).<br />

S8: To make an investigati<strong>on</strong> while in order to choose <strong>on</strong>e from many alternatives.<br />

S9: To make a c<strong>on</strong>clusi<strong>on</strong> and evaluati<strong>on</strong> of the EA.<br />

With the main works included in Table 1, it is important to realize the distinctive capabilities and<br />

applicability of the six types of activities in doing the tasks. We could make a reflecti<strong>on</strong> about how<br />

each type of activity could be applied in each stage of the EA process (Table 4). Based <strong>on</strong> the<br />

scenarios listed and Table 4, we can make a mapping (Table 5) to show how the six types of activities<br />

could be applied. The process to make the mapping is qualitatively based <strong>on</strong>:<br />

Epistemology and pers<strong>on</strong>al experiences. In several EA projects, it was needed to do EA analysis<br />

with some specific approaches and c<strong>on</strong>crete requirements.<br />

Informal discussi<strong>on</strong> with some practiti<strong>on</strong>ers in the domain of EA. Actually, it is interesting and<br />

helpful to communicate with practiti<strong>on</strong>ers. Some of them are employees in industrial companies;<br />

some of them are independent c<strong>on</strong>sultants.<br />

The applicability of scenarios in EA process. The differences of tasks in stages in the EA process<br />

enable practiti<strong>on</strong>ers to identify the different applicability of scenarios in EA process.<br />

The applicability of the six types of activities in the scenarios and EA process. The six types of<br />

activities have different applicability and problem-solving capabilities in the scenarios and the<br />

tasks in stages in EA process.<br />

Table 4: Compulsory or opti<strong>on</strong>al in EA process<br />

Prepare Create Check Acti<strong>on</strong><br />

System thinking Compulsory Compulsory Compulsory Compulsory<br />

Modeling N/A Compulsory Opti<strong>on</strong>al Opti<strong>on</strong>al<br />

Measuring N/A Opti<strong>on</strong>al Compulsory Opti<strong>on</strong>al<br />

Satisfying N/A Opti<strong>on</strong>al Opti<strong>on</strong>al Opti<strong>on</strong>al<br />

Comparing with<br />

requirements<br />

Opti<strong>on</strong>al Opti<strong>on</strong>al Opti<strong>on</strong>al Opti<strong>on</strong>al<br />

Comparing alternatives N/A N/A Opti<strong>on</strong>al Opti<strong>on</strong>al<br />

Table 5: Distributi<strong>on</strong> situati<strong>on</strong> of types of activities with scenarios<br />

5. C<strong>on</strong>clusi<strong>on</strong>s<br />

S1 S2 S3 S4 S5 S6 S7 S8 S9<br />

System thinking I I I I I I I I I<br />

Modeling I I I III III II II IV II<br />

Measuring III II I I II II II II I<br />

Satisfying IV III III III I III III III III<br />

Comparing with requirement IV II III II III IV I IV III<br />

Comparing alternatives V V III III III V IV V III<br />

For scenarios(S1-S9), the classifying system:<br />

I: Should be included, undoubtedly<br />

II: Normally included, if included, it would be more helpful and reas<strong>on</strong>able<br />

III: Sometimes included and sometimes not, it depends <strong>on</strong> the actual c<strong>on</strong>text<br />

IV: Normally not included, if not included, it would be more helpful or reas<strong>on</strong>able<br />

V: Should not be included, undoubtedly<br />

This paper presented six types of activities to characterize EA analysis. A particular EA analysis<br />

approach embraces <strong>on</strong>ly part (<strong>on</strong>e or several) of the six types of activities, not including all the six<br />

340


Haining Wan and Sven Carlss<strong>on</strong><br />

types of activities. More often, EA analysis approaches are c<strong>on</strong>fined to requirements and even<br />

particular organizati<strong>on</strong>s. As a result, practically, there is no EA analysis approach which is so-called<br />

heavyweight engineering <strong>on</strong>e and could be applied in different situati<strong>on</strong>, c<strong>on</strong>text, and background. In<br />

each stage in the EA process, EA analysis has different focus, and the challenge to portray these<br />

different EA analysis calls for in-depth recogniti<strong>on</strong> of the EA process. In order to avoid the c<strong>on</strong>fusi<strong>on</strong><br />

of EA analysis, we propose that finite types of activities could be abstracted from the EA process.<br />

These activities are intuitive, near orthog<strong>on</strong>al and reflecting the nature of system analysis. The result<br />

of the debate whether or not and to which extent these types of activities are involved could help to<br />

define and specify the feature of the c<strong>on</strong>crete EA analysis. This paper tries to facilitate further<br />

research by thinking and rethinking about the term of EA analysis. It is hoped that this paper would<br />

move forward the practices of EA practiti<strong>on</strong>ers; especially mappings from EA analysis task to analysis<br />

approaches and software tools. The illustrati<strong>on</strong>s part of the paper is a little weak. Further research of<br />

this paper would be to do some empirical study with data collecti<strong>on</strong> around the EA lifecycle process,<br />

e.g. to identify the success factors in EA process, to observe the EA analysis as a systems<br />

engineering process, to illustrate how each of the six types of activities could support EA analysis.<br />

References<br />

Buckl, S. (2011) "A Meta-language for Enterprise Architecture Analysis", in Enterprise, Business-Process and<br />

Informati<strong>on</strong> Systems Modeling: 12th Internati<strong>on</strong>al <str<strong>on</strong>g>C<strong>on</strong>ference</str<strong>on</strong>g>, BPMDS 2011, and 1<str<strong>on</strong>g>6th</str<strong>on</strong>g> Internati<strong>on</strong>al<br />

<str<strong>on</strong>g>C<strong>on</strong>ference</str<strong>on</strong>g>, EMMSAD 2011, held at CAiSE 2011, L<strong>on</strong>d<strong>on</strong>, UK, June 20-21, 2011. Proceedings, vol. 81, pp<br />

511-25.<br />

Checkland, P. (1981) Systems thinking, systems practice, J. Wiley.<br />

de Boer, F.S., B<strong>on</strong>sangue, M.M., Jacob, J., Stam, A. & van der Torre, L. (2005) "Enterprise Architecture Analysis<br />

with XML", in Proceedings of the 38th Annual Hawaii Internati<strong>on</strong>al <str<strong>on</strong>g>C<strong>on</strong>ference</str<strong>on</strong>g> <strong>on</strong> System Sciences, 2005.<br />

HICSS '05.<br />

DoD Architecture Framework Working Group. (2003) DoD Architecture Framework Versi<strong>on</strong> 1.0.<br />

DoD Architecture Framework Working Group. (2009) DoD Architecture Framework Versi<strong>on</strong> 2.0.<br />

Johns<strong>on</strong>, P., Lagerström, R., Närman, P. & Sim<strong>on</strong>ss<strong>on</strong>, M. (2007) "Enterprise architecture analysis with extended<br />

influence diagrams", Informati<strong>on</strong> Systems Fr<strong>on</strong>tiers, vol. 9, no. 2, pp 163-80.<br />

Khoury, G.R. & Simoff, S.J. (2004) "Enterprise architecture modelling using elastic metaphors", paper presented<br />

to Proceedings of the first Asian-Pacific c<strong>on</strong>ference <strong>on</strong> C<strong>on</strong>ceptual modelling - Volume 31, Dunedin, New<br />

Zealand.<br />

Lagerström, R., Johns<strong>on</strong>, P. & Hook, D. (2010) "Architecture analysis of enterprise systems modifiability - Models,<br />

analysis, and validati<strong>on</strong>", Journal of Systems and Software, vol. 83, no. 8, pp 1387-403.<br />

Maier, M.W., Emery, D. & Hilliard, R. (2004) "ANSI/IEEE 1471 and systems engineering", Systems Engineering,<br />

vol. 7, no. 3, pp 257-70.<br />

Mayo, D. & Tiemann, M. (2005) "EA: It's not just for IT anymore", Journal of Enterprise Architecture, vol. 1, no. 1,<br />

pp 36-44.<br />

McFadden, T.G. (2001) "Understanding the Internet: Model, metaphor, and analogy", Library Trends, vol. 50, no.<br />

1, pp 87.<br />

Meyers, M. (2011) "Rethinking our Enterprise Architecture Principles", Journal of Enterprise Architecture, vol. 7,<br />

no. 3, pp 41-8.<br />

Moen, R. & Norman, C. (2006) "Evoluti<strong>on</strong> of the PDCA Cycle. Society", pp 1-11.<br />

OpenGroup. (2003) TOGAF 8.1.<br />

OpenGroup. (2010) TOGAF 9.0.<br />

Razavi, M., Aliee, F.S. & Badie, K. (2011) "An AHP-based approach toward enterprise architecture analysis<br />

based <strong>on</strong> enterprise architecture quality attributes", Knowledge and Informati<strong>on</strong> Systems, vol. 28, no. 2, pp<br />

449-72.<br />

Richm<strong>on</strong>d, B. (1987) "System Dynamics/Systems Thinking: Let's Just Get On With It", paper presented to 1994<br />

Internati<strong>on</strong>al Systems Dynamics <str<strong>on</strong>g>C<strong>on</strong>ference</str<strong>on</strong>g>, Sterling, Scotland.<br />

Sessi<strong>on</strong>s, R. 2007, "A Comparis<strong>on</strong> of the Top Four Enterprise-Architecture Methodologies", viewed December 29,<br />

2011, .<br />

The Chief Informati<strong>on</strong> Officers Council. (1999) Federal Enterprise Architecture Framework Versi<strong>on</strong> 1.1.<br />

US Department of the Treasury Chief Informati<strong>on</strong> Officer Council. (2000) Treasury Enterprise Architecture<br />

Framework(Versi<strong>on</strong> 1).<br />

Webnova Inc. 2009, Executable Architecture: Design Must Be Incremental, viewed April 15 2012,<br />

http://www.webnova.ca/Home/ViewPost/5505477172482062627.<br />

Zachman, J.A. (1987) "A framework for informati<strong>on</strong> systems architecture.", IBM Systems Journal, vol. 26, no. 3.<br />

Zachman, J.A. (2001) The Zachman Framework.<br />

Zachman, J.A. (2004) The Zachman Framework 2.<br />

Zachman, J.A. (2011) The Zachman Framework for Enterprise Architecture Versi<strong>on</strong> 3.<br />

Zia, M.J., Azam, F. & Allauddin, M. (2011) "A Survey of Enterprise Architecture Analysis Using Multi Criteria<br />

Decisi<strong>on</strong> Making Models (MCDM)", in R Chen (ed.), Intelligent Computing and Informati<strong>on</strong> Science, Pt Ii,<br />

Springer-Verlag Berlin, Berlin, vol. 135, pp 631-7.<br />

341


Moving Towards a Sensor-Based Patient M<strong>on</strong>itoring<br />

System: Evaluating its Impact <strong>on</strong> Data and Informati<strong>on</strong><br />

Quality<br />

Atieh Zarabzadeh, John O’D<strong>on</strong>oghue, Frederic Adam, Mervyn O’C<strong>on</strong>nell,<br />

Siobhán O’C<strong>on</strong>nor, Sim<strong>on</strong> Woodworth, Joe Gallagher and Tom O’Kane<br />

Health Informati<strong>on</strong> System Research Centre, University College Cork<br />

zarabzadeh@ucc.ie<br />

zarabzaa@tcd.ie<br />

Abstract: For future healthcare systems to take advantage of sensor-based patient m<strong>on</strong>itoring devices, careful<br />

c<strong>on</strong>siderati<strong>on</strong> must be taken in respect to the impact they can have <strong>on</strong> existing workflow processes. In<br />

associati<strong>on</strong> with this, data and informati<strong>on</strong> quality dimensi<strong>on</strong>s need to be incorporated to help ensure a<br />

successful outcome. This paper will explore the utilisati<strong>on</strong> of a paper-based patient assessment scorecard and<br />

the transiti<strong>on</strong> to an electr<strong>on</strong>ic versi<strong>on</strong> with a view to the future adopti<strong>on</strong> of sensor-based devices. To evaluate the<br />

transiti<strong>on</strong> from paper-based to sensor-based soluti<strong>on</strong>s, the Modified Early Warning Scorecard (MEWS) is the<br />

primary exemplar within this paper. MEWS has a defined set of protocols and guidelines that assist the<br />

healthcare providers in classifying a patient’s status of health in detecting patient deteriorati<strong>on</strong>. Paper-based<br />

MEWS are already deployed within Medical Assessment Units (MAU). Thus, the MAU is an ideal test bed for the<br />

evaluati<strong>on</strong> of sensor-based soluti<strong>on</strong>s. The Socio-Technical Informati<strong>on</strong> Systems Design (STISD) science<br />

research framework is the methodology employed to address a practical problem (i.e. frequent capturing of<br />

patient vital signs) raised by the healthcare providers in relati<strong>on</strong> to MEWS. In accordance with the STISD<br />

framework, a review of the extant theories, knowledge and data reveal an Event-driven Process Chain (EPC)<br />

diagram for the paper-based MEWS. Based <strong>on</strong> these findings an electr<strong>on</strong>ic versi<strong>on</strong> of the paper-based MEWS<br />

(eMEWS) is proposed and tested. Further refinements are required to explore the full capabilities of sensorbased<br />

soluti<strong>on</strong>s and the role they can play within MEWS. Presented is a c<strong>on</strong>ceptual model which examines the<br />

relati<strong>on</strong>ship between the c<strong>on</strong>tent and system quality measures and their associated independent variables. Alpha<br />

and gamma tests are c<strong>on</strong>ducted to evaluate the eMEWS against the desired outcomes and provide the<br />

foundati<strong>on</strong> for the development of the sensor-based eMEWS soluti<strong>on</strong>. Results show that the eMEWS prototype<br />

addresses key data quality dimensi<strong>on</strong>s, with sensor-based eMEWS indicting potential enhancement to the<br />

timeliness and frequency of data capturing.<br />

Keywords: sensor-based electr<strong>on</strong>ic modified early warning scorecard, data and informati<strong>on</strong> quality, sociotechnical<br />

informati<strong>on</strong> systems design methodology<br />

1. Introducti<strong>on</strong><br />

Electr<strong>on</strong>ic healthcare systems are found to improve the patient care delivery process by reducing<br />

medicati<strong>on</strong> error (Bates 2000; Kaushal et al. 2001) and as a c<strong>on</strong>sequence, improving the quality of<br />

care (Chaudhry et al. 2006; Koppar & Sridhar 2009). However, little research has been c<strong>on</strong>ducted <strong>on</strong><br />

improving the process of assessing patient health status in Medical Assessment Unit (MAU) where<br />

the patients with various health statuses are classified using Modified Early Warning Scorecard<br />

(MEWS) protocols. Thus, this paper focuses <strong>on</strong> improving patient outcomes in a MAU ward by<br />

exploring the impact of data and informati<strong>on</strong> quality dimensi<strong>on</strong>s. To serve this purpose, influential<br />

data quality factors <strong>on</strong> patient outcomes are identified and discussed. The Socio-Technical<br />

Informati<strong>on</strong> Systems Design (STISD) science (Carlss<strong>on</strong> et al. 2011) methodology is employed to<br />

provide a design theory c<strong>on</strong>cerning improvements <strong>on</strong> paper-based MEWS. An electr<strong>on</strong>ic Modified<br />

Early Warning Scorecard (eMEWS) is proposed, which is then extended to a sensor-based MEWS to<br />

meet the desired outcomes. Abnormalities in vital sign readings are found to indicate the risk of<br />

adverse clinical events in the state of the health of the patient up to several hours before the<br />

unexpected event (Kause et al. 2004; Hillman et al. 2002; Jacques et al. 2006; Gustafs<strong>on</strong> et al. 1999).<br />

These adverse events may lead to avoidable and unexpected death of the patient or poor patient<br />

outcome (McGloin et al. 1999). Studies revealed that improvements <strong>on</strong> the detecti<strong>on</strong> of such<br />

abnormalities and timely acti<strong>on</strong>s are likely to provide avenues to substantially reduce the risk of rapid<br />

deteriorati<strong>on</strong> in patients’ c<strong>on</strong>diti<strong>on</strong> whilst in hospital (Buist et al. 2004; Sax & Charls<strong>on</strong> 1987). Paperbased<br />

MEWS are used to classify patients with regards to the likelihood of deteriorati<strong>on</strong> or adverse<br />

event (Subbe et al. 2001; Gardner-Thorpe et al. 2006) and score them accordingly. MEWS is a set of<br />

guidelines and protocols for interpreting vital sign readings. Each vital sign parameter is categorised<br />

into bands which are associated with a score. The sum of the scores for a set of vital sign reading is<br />

the MEWS score for the patient at that point of time (O’Kane et al. 2010). High MEWS scores<br />

dem<strong>on</strong>strate the likelihood of patient deteriorati<strong>on</strong> occurring (G. B. Smith et al. 2006). Thus MEWS<br />

342


Atieh Zarabzadeh et al.<br />

serves as a paper based Clinical Decisi<strong>on</strong> Support System (CDSS) by utilising knowledge and<br />

informati<strong>on</strong> presentati<strong>on</strong> to healthcare providers in accordance with the MEWS protocols and<br />

guidelines in a timely manner (Berner 2009). Secti<strong>on</strong> 2 of this paper describes the Socio-Technical<br />

Informati<strong>on</strong> Systems Design (STISD) methodology employed in the presented research. This paper<br />

explores three approaches to MEWS: paper-based MEWS, eMEWS, and sensor-based eMEWS.<br />

Secti<strong>on</strong> 3 describes the relevant activity to problem situati<strong>on</strong> identificati<strong>on</strong> and desired outcomes in<br />

relati<strong>on</strong> to paper-based MEWS. Healthcare providers are interviewed and visits to a MAU ward are<br />

c<strong>on</strong>ducted to establish the ground for proposing a design theory which is the proposed eMEWS. The<br />

extant knowledge and data is presented in Secti<strong>on</strong> 4. A design theory is then proposed and described<br />

comprehensively in Secti<strong>on</strong> 5. The design theory is tested by alpha and gamma testing methods with<br />

results presented in Secti<strong>on</strong> 6. Alpha testing is carried out by the original developers of the system,<br />

while gamma testing is c<strong>on</strong>ducted by the users of the system. A refinement to the proposed design is<br />

then described to incorporate the sensor-based eMEWS in Secti<strong>on</strong> 7.<br />

2. Methodology<br />

The Socio-Technical Informati<strong>on</strong> Systems Design (STISD) science research framework developed by<br />

(Carlss<strong>on</strong> et al. 2011) is based <strong>on</strong> Informati<strong>on</strong> Systems (IS) design science (Hevner et al. 2004;<br />

Peffers et al. 2006) and management design science (Van Aken 2005). Table 1 represents the four<br />

main activities of the STISD framework and how this paper relates to each of them, noting the<br />

numbered sequence of each activity. The problem targeted by this paper is experienced by the<br />

healthcare providers in their practice. As dem<strong>on</strong>strated in Table 1, the STISD framework is a suitable<br />

and appropriate theory for the objectives of this research. While requiring the design theory to be<br />

established <strong>on</strong> solid grounds, STISD framework allows for multiple iterative refinements to a design<br />

theory and supports a variety of testing methods.<br />

Table 1: The proposed research mapped to the STISD activities (Carlss<strong>on</strong> et al. 2011)<br />

STISD Activities Activity Adopti<strong>on</strong> by the Research<br />

Identify problem<br />

situati<strong>on</strong>s and<br />

desired outcomes<br />

Review extant<br />

theories, knowledge<br />

and data<br />

Propose/refine<br />

design theory and<br />

knowledge<br />

Test design theory<br />

and knowledge<br />

1) The problem identified in this research has practical implicati<strong>on</strong>s for the<br />

practiti<strong>on</strong>ers in the area. The desired outcomes identified are to match the problem.<br />

2) To gather existing research and knowledge, healthcare providers are interviewed<br />

and site visits are c<strong>on</strong>ducted. The focus of these activities is to achieve the desired<br />

outcomes.<br />

3) A design theory is proposed and described comprehensively to c<strong>on</strong>textualise the<br />

problem.<br />

5) The proposed design theory is refined and described to the practiti<strong>on</strong>ers in the<br />

area.<br />

4) The proposed design theory is evaluated using alpha and gamma approaches.<br />

The results of the evaluati<strong>on</strong> derived the refined design theory.<br />

The process followed in the presented research is adopted from the STISD framework and is<br />

presented in Figure 1. The problem situati<strong>on</strong>s and desired outcomes are identified. Existing theories,<br />

knowledge and data are reviewed to establish the grounds for the design. On this basis, a design<br />

theory, i.e. eMEWS prototype, is proposed and tested. Refined design theory and knowledge, i.e.<br />

sensor-based eMEWS, is then developed and is subjected to tests in the future iterati<strong>on</strong>s. Further<br />

refinements may also be deemed necessary.<br />

Identify Problem Situati<strong>on</strong>s<br />

and Desired Outcomes (Secti<strong>on</strong> 3)<br />

Propose Design Theory and<br />

Knowledge (Secti<strong>on</strong> 5)<br />

Refine Design Theory and<br />

Knowledge (Secti<strong>on</strong> 7)<br />

Review Extant Theories, Knowledge<br />

and Data (Secti<strong>on</strong> 4)<br />

Test Design Theory and<br />

Knowledge (Secti<strong>on</strong> 6)<br />

Test Design Theory and<br />

Knowledge (Future Work)<br />

Figure 1: STISD research framework applied to MEWS adapted from (Carlss<strong>on</strong> et al. 2011)<br />

343


Atieh Zarabzadeh et al.<br />

3. Identify problem situati<strong>on</strong> and desired outcomes<br />

The aim of this research is to facilitate improvement <strong>on</strong> the patient outcomes in the MAU ward.<br />

Improvements in the patient outcomes require improvements in the healthcare providers’ decisi<strong>on</strong><br />

making process. At a high level this process involves measuring patients’ vital signs, calculating their<br />

MEWS score, interpreting both these scores and other signs, not amenable to capture in a scorecard,<br />

making a decisi<strong>on</strong> and taking appropriate acti<strong>on</strong>(s). Building <strong>on</strong> (O’C<strong>on</strong>nor et al. 2011) two measures<br />

are identified that underpin the proposed model as presented in Figure 2:<br />

C<strong>on</strong>tent quality: involves the informati<strong>on</strong> and data quality exclusive of the technology used.<br />

System quality: determined by the technologies used.<br />

Accessibility<br />

Completeness<br />

C<strong>on</strong>cise Representati<strong>on</strong><br />

C<strong>on</strong>sistent Representati<strong>on</strong><br />

Ease of Manipulati<strong>on</strong><br />

Free-of-error<br />

Objectivity<br />

Relevancy<br />

Timeliness<br />

Understandability<br />

C<strong>on</strong>tent Quality Independent Variables<br />

Improved Patient Outcomes<br />

Scope and Scale of<br />

Measurements<br />

Frequency of reporting<br />

Measurements<br />

N<strong>on</strong>-vital sign Informati<strong>on</strong><br />

System Quality Independent Variables<br />

Figure 2: Independent variables c<strong>on</strong>tributing to improved patient outcomes<br />

The c<strong>on</strong>tent quality measure that impacts the patient outcomes includes ten data and informati<strong>on</strong><br />

quality dimensi<strong>on</strong>s relevant to MEWS as shown in Table 2 (Pipino et al. 2002).<br />

The system quality measure that impacts the patient outcome includes three independent variables:<br />

Scope and scale of measurements: The extent to which vital sign data and the scales used for<br />

taking measurements reflect the patient health status.<br />

Frequency of reporting measurements: Infrequent data reporting and MEWS score calculati<strong>on</strong><br />

may lead to the lack of sufficient data. Appropriate frequency of data capturing may assist in<br />

preventative timely acti<strong>on</strong>s in the MAU ward.<br />

N<strong>on</strong>-vital sign informati<strong>on</strong>: The subjective data being collected by the healthcare providers at the<br />

point of care delivery and through various methods including observati<strong>on</strong> and c<strong>on</strong>versati<strong>on</strong> with<br />

the patient may c<strong>on</strong>tribute to the decisi<strong>on</strong> making process as a valid determinant. Crucially, these<br />

signs are not amenable to coding in a scorecard.<br />

In this paper, the focus is <strong>on</strong> c<strong>on</strong>cise and c<strong>on</strong>sistent representati<strong>on</strong>, free-of-error, timeliness and<br />

understandability of the c<strong>on</strong>tent measure. Therefore, the desired outcome of this research is a<br />

method that meets these data and informati<strong>on</strong> quality parameters. The authors propose that the<br />

model underpinning this research requires a computer-based design theory that eliminates the<br />

manual process of taking measurements and calculating MEWS score.<br />

Table 2: Data and informati<strong>on</strong> quality parameters (Pipino et al. 2002) and their relevancy to MEWS<br />

Parameters<br />

(Pipino et al.<br />

2002)<br />

Accessibility<br />

Completeness<br />

C<strong>on</strong>cise<br />

representati<strong>on</strong><br />

Relevance to MEWS<br />

Availability and accessibility of patient data at any given time is a critical requirement in<br />

MEWS<br />

Vital sign dataset needs to be complete in order to obtain the real MEWS score. In additi<strong>on</strong><br />

to vital signs, other subjective data gathered via observati<strong>on</strong> or c<strong>on</strong>versati<strong>on</strong> should be<br />

c<strong>on</strong>sidered.<br />

C<strong>on</strong>cise and brief representati<strong>on</strong> of data is very important to reduce the time spent reading<br />

data and increase the time spent <strong>on</strong> care delivery.<br />

344


Parameters<br />

(Pipino et al.<br />

2002)<br />

C<strong>on</strong>sistent<br />

representati<strong>on</strong><br />

Ease of<br />

i l i<br />

Free-of-error<br />

Atieh Zarabzadeh et al.<br />

Relevance to MEWS<br />

With the large quantity of data being presented in the MEWS, c<strong>on</strong>sistency in the<br />

representati<strong>on</strong> of data is critical.<br />

Vital sign data gathered should be easily applied to various activities as part of the decisi<strong>on</strong><br />

ki<br />

Decisi<strong>on</strong>s made are extremely sensitive. Thus vital sign data must be correct, reliable and<br />

Objectivity The explicit (i.e. vital sign) and subjective data must be impartial.<br />

Relevancy<br />

Timeliness<br />

Vital signs being c<strong>on</strong>sidered for MEWS are already proven relevant to the decisi<strong>on</strong> making<br />

process. The subjective data needs to be incorporated to the decisi<strong>on</strong> making process.<br />

Real time vital sign data is of the utmost importance in the c<strong>on</strong>text of MEWS where patients<br />

are under assessment and their status of health may change rapidly.<br />

Understandabili Comprehensi<strong>on</strong> of vital sign data is critical to MEWS.<br />

To test the proposed system against the desired outcomes, the following testing strategies are<br />

applied:<br />

C<strong>on</strong>cise and c<strong>on</strong>sistent representati<strong>on</strong> is achieved with an electr<strong>on</strong>ic system that uses the same<br />

format to display data.<br />

Free-of-error is tested by an alpha method where the developers of the design assess if the<br />

proposed design has allowed for sufficient automati<strong>on</strong> in capturing vital sign data, calculating the<br />

MEWS score and selecting an appropriate acti<strong>on</strong> protocol.<br />

Timeliness is obtained as a product of full automati<strong>on</strong> (i.e. sensors) of the data capturing process<br />

with little or no human interventi<strong>on</strong>.<br />

Understandability of the design theory is tested by a gamma method where the healthcare<br />

providers assess their comprehensi<strong>on</strong> of the data presented in the c<strong>on</strong>cise and c<strong>on</strong>sistent<br />

fashi<strong>on</strong>.<br />

Previous attempts at the adopti<strong>on</strong> of new systems in the healthcare sector have faced adopti<strong>on</strong><br />

challenges (Lluch 2011; Hikmet et al. 2008). Thus, the adopti<strong>on</strong> of the design theory needs a<br />

thorough assessment achieved by gamma testing.<br />

The presented paper evaluates the design theory by:<br />

Alpha testing:<br />

The design theory developers assess the c<strong>on</strong>ciseness and c<strong>on</strong>sistency of the representati<strong>on</strong> of<br />

data.<br />

The original developers ensure the system is eliminating or reducing the sources of error.<br />

Testing the timeliness parameter may entail testing a full automati<strong>on</strong> that can be carried out by<br />

the original developers.<br />

Gamma testing:<br />

Understandability can be tested through a survey with healthcare providers.<br />

Given the importance of adopti<strong>on</strong> of new system or technology in the healthcare setting, a test is<br />

carried out by the healthcare providers commenting <strong>on</strong> the adopti<strong>on</strong> challenges they foresee.<br />

4. Review extant theories, knowledge and data<br />

The design theory has to be grounded <strong>on</strong> previous theories, knowledge or data. In the c<strong>on</strong>text of the<br />

presented paper, the extant theory and knowledge relates to the existing process diagram for the<br />

paper-based MEWS. The Event-driven Process Chain (EPC) diagram for the paper-based MEWS is<br />

presented in Figure 3 has the following steps: 1) the patient arrival to the ward, 2) vital sign<br />

measurements taken, 3) MEWS score calculated, 4) a decisi<strong>on</strong> made, and 5) an associated acti<strong>on</strong><br />

taken. Figure 3 is c<strong>on</strong>structed <strong>on</strong> the observati<strong>on</strong>s from visits to the MAU ward and interviews with<br />

healthcare providers in this ward.<br />

345


Atieh Zarabzadeh et al.<br />

Figure 3: MAU EPC diagram for MEWS<br />

An overview of the EPC diagram: The patient medical assessment process (the focus of this paper)<br />

including MEWS 1 commences when a patient arrives to the MAU and terminates <strong>on</strong> patient leaving<br />

this ward. A patient may be admitted from the hospital recepti<strong>on</strong>, Accident and Emergency (A&E), a<br />

General Practiti<strong>on</strong>er (GP) or an ambulance. Prior to MEWS score calculati<strong>on</strong>, a series of background<br />

questi<strong>on</strong>s are asked from the patient. Vital signs may also be captured. Vital signs are manually<br />

measured and transcribed <strong>on</strong> the patient chart. MEWS score is then manually calculated and an<br />

acti<strong>on</strong> protocol is identified. Based <strong>on</strong> the proposed acti<strong>on</strong>, a decisi<strong>on</strong> is made. Nurses may transcribe<br />

additi<strong>on</strong>al observati<strong>on</strong>s and data they gather by talking to the patient. These notes may be c<strong>on</strong>sulted<br />

1<br />

The process that the patient undergoes in the MAU ward is <strong>on</strong>e stage of a multi stage care delivery process. Other stages of<br />

care delivery process are not the subject of this paper.<br />

346


Atieh Zarabzadeh et al.<br />

at the time of decisi<strong>on</strong> making or may be referred to other healthcare providers. An acti<strong>on</strong> is taken if<br />

needed; otherwise the need for further vital sign measurement is assessed. The process is completed<br />

<strong>on</strong>ce the healthcare provider decides that no more vital sign measurements are needed. The patient<br />

may be discharged from the hospital, admitted to the hospital. The MEWS protocols and guidelines<br />

play a crucial role in classifying a patient’s health status. There are two sets of procedures c<strong>on</strong>cerning<br />

MEWS in the patient assessment process as highlighted in Figure 3: 1) recording vital signs, and 2)<br />

evaluating the frequency of measurements.<br />

5. Propose design theory and knowledge<br />

To achieve the desired outcomes, an electr<strong>on</strong>ic versi<strong>on</strong> of the paper-based MEWS is designed and<br />

developed (eMEWS). The proposed design aims at addressing the independent variables described<br />

in the previous secti<strong>on</strong>. Thus, an eMEWS prototype is developed based <strong>on</strong> the design theory and the<br />

paper-based MEWS. eMEWS prototype essentially follows the paper-based MEWS forms to maintain<br />

c<strong>on</strong>ciseness and c<strong>on</strong>sistency. However, additi<strong>on</strong>al features such as colour coded trends are<br />

integrated (Zarabzadeh et al. 2012) to enhance the users’ experience. The eMEWS prototype is<br />

presented to the healthcare providers who work in the MAU and their comments <strong>on</strong> how it may impact<br />

their daily workload are explored.<br />

The workflow presented in Figure 3 was revised to take <strong>on</strong> board the features of eMEWS. The circled<br />

activities labelled as “Recording vital signs” are automated to reduce transcribing and calculati<strong>on</strong><br />

errors as shown in Figure 4. All other activities remain the same. With the proposed design theory,<br />

the vital signs are taken by the healthcare provider and are typed using the tablet PC soluti<strong>on</strong>. The<br />

MEWS score is auto-calculated and presented to the user in a c<strong>on</strong>sistent fashi<strong>on</strong>.<br />

Figure 4: eMEWS refinement to the workflow (updated “Recording vital signs” from Figure 3)<br />

6. Test design theory and knowledge<br />

According to the STISD framework the proposed design is subject to testing. The testing<br />

methodologies used are alpha and gamma, as described:<br />

6.1 Alpha testing<br />

Testing the c<strong>on</strong>ciseness and c<strong>on</strong>sistency of the data representati<strong>on</strong>: The developers of the<br />

eMEWS prototype propose that the system is c<strong>on</strong>sistent in recording vital of signs and in<br />

representing the data collected. The user interfaces are c<strong>on</strong>cise and clear in representing data.<br />

This has been validated through a number of internal static evaluati<strong>on</strong>s.<br />

Reducing the sources of error has been achieved to a certain extent. In the eMEWS prototype<br />

where the users type the measurements, the system calculates the MEWS score, identifies and<br />

presents the appropriate acti<strong>on</strong> protocol. The design is still pr<strong>on</strong>e to typing errors which can be<br />

addressed in the refined design. This finding c<strong>on</strong>curs with (M. Mohammed et al. 2009) the<br />

accuracy of calculating scores has improved by running the scorecard <strong>on</strong> the hand-held<br />

computers.<br />

347


Atieh Zarabzadeh et al.<br />

The developers of the design theory believe that the timeliness parameter is not fully addressed in<br />

the proposed theory. However, eMEWS increases the speed of data transcripti<strong>on</strong> which c<strong>on</strong>curs<br />

with (D. R. Prytherch et al. 2006).<br />

Thus, while the c<strong>on</strong>cise and c<strong>on</strong>sistent data representati<strong>on</strong> is facilitated by the eMEWS proposed<br />

design theory, further refinement is needed to eliminate the sources of error and allow a real-time upto-date<br />

data capturing strategy.<br />

6.2 Gamma testing<br />

A survey was c<strong>on</strong>ducted and completed by 71 participants of whom 11 were medical doctors, 56 were<br />

nurses, and the remaining four did not specify their expertise. All participants are based in the same<br />

hospital. Participants’ nursing experience and their prior knowledge of MEWS were recorded.<br />

Approximately 65% of the resp<strong>on</strong>dents had over 10 years of nursing experience. Of the 66 who<br />

resp<strong>on</strong>ded to the questi<strong>on</strong> <strong>on</strong> their prior experience with paper-based MEWS, <strong>on</strong>ly 24 had more than<br />

24 m<strong>on</strong>ths of MEWS experience. Thus, majority of the participants are senior nurses and have prior<br />

exposure to paper-based MEWS.<br />

Understandability of the data: 62 out of 67 resp<strong>on</strong>dents find the data in the eMEWS easy or very easy<br />

to understand. Five out of 67 are neutralas presented Table 3 (Zarabzadeh et al. 2012).<br />

Table 3: Participants’ resp<strong>on</strong>ses to understandability of data represented by eMEWS<br />

Opti<strong>on</strong>s Rep<strong>on</strong>ses % of resp<strong>on</strong>ses<br />

Very easy to comprehend 18 27%<br />

Easy to comprehend 44 66%<br />

Neither easy nor difficult to comprehend 5 7%<br />

Difficult to comprehend 0 0%<br />

Very difficult to comprehend 0 0%<br />

To evaluate the adopti<strong>on</strong> challenges of the design theory, after a dem<strong>on</strong>strati<strong>on</strong> of the applicati<strong>on</strong>,<br />

survey participants were asked if they “foresee any issues with the eMEWS fitting in with the existing<br />

ward practices”. A number of resp<strong>on</strong>dents did not foresee any challenges in adopti<strong>on</strong> of eMEWS, for<br />

instance <strong>on</strong>e commented “...it should fit in with existing ward practices”. Others made the following<br />

comments:<br />

Double documentati<strong>on</strong>: the doctors will need paper records and print outs, which will impose time<br />

c<strong>on</strong>straints.<br />

Lack of resources: providing a computer m<strong>on</strong>itor for each bed can be challenging both financially<br />

and space-wise. Technical support is also needed.<br />

Staff training: different staff may initially find working with the new system difficult, so prior training<br />

is required.<br />

Building trust in technology which may break down. Regular back up of the database may assist<br />

in establishing c<strong>on</strong>fidence in the technology am<strong>on</strong>g the staff. One comment envisaged the<br />

“maintenance of machine could be a problem”.<br />

Security issues: which may include technical security c<strong>on</strong>cerns and doctors access to all patients’<br />

data <strong>on</strong> the ward.<br />

In additi<strong>on</strong> to the presented issues that the participants raised, comments were made <strong>on</strong> the benefits<br />

of the eMEWS. For example, <strong>on</strong>e comment c<strong>on</strong>firmed that automated MEWS score calculati<strong>on</strong><br />

“should be beneficial for nursing staff not to have to calculate [MEWS score]”.<br />

7. Refine design theory and knowledge<br />

A refinement to the proposed design theory is anticipated to address the inefficiencies c<strong>on</strong>cerning 1)<br />

sources of error, and 2) real-time up-to-date vital signs data capturing revealed by alpha testing in<br />

previous secti<strong>on</strong>.<br />

The refinement can be made to the eMEWS prototype by the adopti<strong>on</strong> of wireless patient m<strong>on</strong>itoring<br />

sensors. A sensor-based eMEWS may improve the speed of the process by facilitating c<strong>on</strong>stant<br />

m<strong>on</strong>itoring of the vital signs and therefore improving the proposed design theory to address timeliness<br />

348


Atieh Zarabzadeh et al.<br />

of collecti<strong>on</strong> and presentati<strong>on</strong> of data. Furthermore, this system eliminates the manual vital sign<br />

recording.<br />

With the proposed update to the “Recording vital signs” comp<strong>on</strong>ent of the workflow (the refinement to<br />

Figure 3 leading to the outcome Figure 4) and the introducti<strong>on</strong> of sensor-based eMEWS, all four<br />

steps are reduced as shown in Figure 5. This sensor-based eMEWS system automatically captures<br />

all patient vital signs and stores the recorded data in a database. In turn the sensor-based eMEWS<br />

automatically calculates and displays the MEWS score with the associated acti<strong>on</strong> protocol to the user.<br />

By providing a c<strong>on</strong>tinuous and customized set of measurements, the sensor-based eMEWS can<br />

reduce a number of the data capturing steps in the previous design theory. Furthermore, the number<br />

of decisi<strong>on</strong> points taken by a health care provider when deciding <strong>on</strong> the frequency of data capture is<br />

reduced. In relati<strong>on</strong> to paper-based MEWS workflow diagram (Figure 3), the sensor-based eMEWS<br />

eliminates the “Evaluating the frequency of measurements” comp<strong>on</strong>ent.<br />

Figure 5: Sensor-based eMEWS improvement to the workflow (updated “Recording vital signs” from<br />

Figure 3)<br />

The refined design (i.e. the sensor-based eMEWS) is subject to further testing in relati<strong>on</strong> to the<br />

independent variables presented in Figure 2. However at theory level it appears promising in terms of<br />

delivering the desired objectives by increasing the frequency of measurements to c<strong>on</strong>tinuous or<br />

customized intervals. At this point, the refinement and testing process are <strong>on</strong>going.<br />

8. Future work<br />

Using the STISD framework a sensor-based eMEWS is designed to allow for c<strong>on</strong>cise and c<strong>on</strong>sistent<br />

data representati<strong>on</strong>, error-free, timely and comprehensible vital sign data capturing. However, further<br />

refinements to the design theory are deemed necessary. The refinements can be carried out <strong>on</strong><br />

multiple areas including facilitating appropriate frequency of reporting vital sign measurements,<br />

clarifying scope and scale of measurements and more importantly from a decisi<strong>on</strong> support viewpoint,<br />

supporting the capture and interpretati<strong>on</strong> of n<strong>on</strong>-vital sign informati<strong>on</strong>.<br />

Acknowledgments<br />

This publicati<strong>on</strong> has emanated from research c<strong>on</strong>ducted with the financial support of Science<br />

Foundati<strong>on</strong> Ireland under Grant Number “SFI 11/RFP.1/CMS/3338”. The authors wish to<br />

acknowledge their gratitude to all the staff at St. Luke’s General Hospital, Kilkenny, for their kind<br />

support in facilitating the workshops and for their <strong>on</strong>going participati<strong>on</strong> in the survey and interviews.<br />

References<br />

Bates, D.W., 2000. Using informati<strong>on</strong> technology to reduce rates of medicati<strong>on</strong> errors in hospitals. BMJ, 320.<br />

Berner, E.S., 2009. Clinical Decisi<strong>on</strong> Support Systems : State of the Art, AHRQ Publicati<strong>on</strong> No. 09-0069-EF.<br />

Rockville, Maryland: Agency for Healthcare Research and Quality.<br />

Buist, M. et al., 2004. Associati<strong>on</strong> between clinically abnormal observati<strong>on</strong>s and subsequent in-hospital mortality:<br />

a prospective study. Resuscitati<strong>on</strong>, 62(2), pp.137-141. Available at:<br />

http://www.sciencedirect.com/science/article/pii/S0300957204001236.<br />

Carlss<strong>on</strong>, S. et al., 2011. Socio-technical IS design science research: developing design theory for IS integrati<strong>on</strong><br />

management. Informati<strong>on</strong> Systems and E-Business <strong>Management</strong>, 9(1), pp.109-131. Available at:<br />

http://dx.doi.org/10.1007/s10257-010-0140-6.<br />

349


Atieh Zarabzadeh et al.<br />

Chaudhry, B. et al., 2006. Systematic Review: Impact of Health Informati<strong>on</strong> Technology <strong>on</strong> Quality, Efficiency,<br />

and Costs of Medical Care. Annals of Internal Medicine, 144(10), pp.742-752. Available at:<br />

http://www.annals.org/c<strong>on</strong>tent/144/10/742.abstract.<br />

Gardner-Thorpe, J. et al., 2006. The value of Modified Early Warning Score (MEWS) in surgical in-patients: a<br />

prospective observati<strong>on</strong>al study. Annals of The Royal College of Surge<strong>on</strong>s of England, 88(6), pp.571-575.<br />

Available at: http://www.ingentac<strong>on</strong>nect.com/c<strong>on</strong>tent/rcse/arcs/2006/00000088/00000006/art00013.<br />

Gustafs<strong>on</strong>, D.H. et al., 1999. Impact of a patient-centered, computer-based health informati<strong>on</strong>/support system.<br />

American journal of preventive medicine, 16(1), pp.1-9. Available at:<br />

http://linkinghub.elsevier.com/retrieve/pii/S0749379798001081?showall=true.<br />

Hevner, A.R. et al., 2004. Design science in informati<strong>on</strong> systems research. MIS Q., 28(1), pp.75-105. Available<br />

at: http://dl.acm.org/citati<strong>on</strong>.cfm?id=2017212.2017217.<br />

Hikmet, N. et al., 2008. The role of organizati<strong>on</strong>al factors in the adopti<strong>on</strong> of healthcare informati<strong>on</strong> technology in<br />

Florida hospitals. Health Care <strong>Management</strong> Science, 11(1), pp.1-9. Available at:<br />

http://dx.doi.org/10.1007/s10729-007-9036-5.<br />

Hillman, K. et al., 2002. Durati<strong>on</strong> of life-threatening antecedents prior to intensive care admissi<strong>on</strong>. Intensive Care<br />

Medicine, 28(11), pp.1629-1634. Available at: http://dx.doi.org/10.1007/s00134-002-1496-y .<br />

Jacques, T. et al., 2006. Signs of critical c<strong>on</strong>diti<strong>on</strong>s and emergency resp<strong>on</strong>ses (SOCCER): A model for predicting<br />

adverse events in the inpatient setting. Resuscitati<strong>on</strong>, 69(2), pp.175-183. Available at:<br />

http://www.sciencedirect.com/science/article/pii/S0300957205003813.<br />

Kause, J. et al., 2004. A comparis<strong>on</strong> of Antecedents to Cardiac Arrests, Deaths and EMergency Intensive care<br />

Admissi<strong>on</strong>s in Australia and New Zealand, and the United Kingdom—the ACADEMIA study. Resuscitati<strong>on</strong>,<br />

62(3), pp.275-282. Available at: http://www.sciencedirect.com/science/article/pii/S0300957204002473.<br />

Kaushal, R., Barker, K.N. & Bates, D.W., 2001. How Can Informati<strong>on</strong> Technology Improve Patient Safety and<br />

Reduce Medicati<strong>on</strong> Errors in Children’s Health Care? Archives of Pediatrics Adolescent Medicine, 155(9),<br />

pp.1002-1007. Available at: http://archpedi.ama-assn.org/cgi/c<strong>on</strong>tent/abstract/155/9/1002.<br />

Koppar, A.R. & Sridhar, V., 2009. A Workflow Soluti<strong>on</strong> for Electr<strong>on</strong>ic Health Records to Improve Healthcare<br />

Delivery Efficiency in Rural India. In eHealth, Telemedicine, and Social Medicine, 2009. eTELEMED ’09.<br />

Internati<strong>on</strong>al <str<strong>on</strong>g>C<strong>on</strong>ference</str<strong>on</strong>g> <strong>on</strong>. pp. 227-232.<br />

Lluch, M., 2011. Healthcare professi<strong>on</strong>als’ organisati<strong>on</strong>al barriers to health informati<strong>on</strong> technologies—A literature<br />

review. Internati<strong>on</strong>al Journal of Medical Informatics, 80(12), pp.849-62. Available at:<br />

http://www.sciencedirect.com/science/article/pii/S1386505611001961.<br />

McGloin, H., Adam, S.K. & Singer, M., 1999. Unexpected deaths and referrals to intensive care of patients <strong>on</strong><br />

general wards. Are some cases potentially avoidable? Journal of the Royal College of Physicians of<br />

L<strong>on</strong>d<strong>on</strong>, 33(3), pp.255-259. Available at: http://cat.inist.fr/?aModele=afficheN&cpsidt=1551867 [Accessed<br />

October 31, 2011].<br />

Mohammed, M., Hayt<strong>on</strong>, R. & Clements, G, 2009. Improving accuracy and efficiency of early warning scores in<br />

acute care. British Journal of Nursing, 18(1), pp.18-24. Available at:<br />

http://elib.tcd.ie/login?url=http://search.proquest.com/docview/764373854?accountid=14404.<br />

O’C<strong>on</strong>nor, Y., O’D<strong>on</strong>oghue, J & O’Reilly, P., 2011. Understanding Mobile Technology Post-Adopti<strong>on</strong> Behaviour:<br />

Impact up<strong>on</strong> Knowledge Creati<strong>on</strong> and Individual Performance. In Mobile Business (ICMB), 2011 Tenth<br />

Internati<strong>on</strong>al <str<strong>on</strong>g>C<strong>on</strong>ference</str<strong>on</strong>g> <strong>on</strong>. pp. 275-282.<br />

O’Kane, T. et al., 2010. MEWS to e-MEWS: From a Paper-Based to an Electr<strong>on</strong>ic Clinical Decisi<strong>on</strong> Support<br />

System. In 4th <str<strong>on</strong>g>European</str<strong>on</strong>g> <str<strong>on</strong>g>C<strong>on</strong>ference</str<strong>on</strong>g> <strong>on</strong> Informati<strong>on</strong> <strong>Management</strong> and Evaluati<strong>on</strong>. Lisb<strong>on</strong>, Portugal.<br />

Peffers, K. et al., 2006. The Design Science Research Process: A Model for Producing and Presenting<br />

Informati<strong>on</strong> Systems Research. In DESRIST. Clarem<strong>on</strong>t, CA, pp. 83-106.<br />

Pipino, L.L., Lee, Y.W. & Wang, R.Y., 2002. Data quality assessment. Commun. ACM, 45(4), pp.211-218.<br />

Available at: http://doi.acm.org/10.1145/505248.506010.<br />

Prytherch, D.R. et al., 2006. Calculating early warning scores—A classroom comparis<strong>on</strong> of pen and paper and<br />

hand-held computer methods. Resuscitati<strong>on</strong>, 70(2), pp.173-178. Available at:<br />

http://www.sciencedirect.com/science/article/pii/S0300957205005526.<br />

Sax, F.L. & Charls<strong>on</strong>, M.E., 1987. Medical patients at high risk for catastrophic deteriorati<strong>on</strong>. Critical care<br />

medicine, 15(5), pp.510-515. Available at: http://ukpmc.ac.uk/abstract/MED/3568715.<br />

Smith, G.B. et al., 2006. Hospital-wide physiological surveillance–A new approach to the early identificati<strong>on</strong> and<br />

management of the sick patient. Resuscitati<strong>on</strong>, 71(1), pp.19-28. Available at:<br />

http://www.sciencedirect.com/science/article/pii/S0300957206001286.<br />

Subbe, C.P. et al., 2001. Validati<strong>on</strong> of a modified Early Warning Score in medical admissi<strong>on</strong>s. QJM, 94(10),<br />

pp.521-526. Available at: http://qjmed.oxfordjournals.org/c<strong>on</strong>tent/94/10/521.abstract .<br />

Van Aken, J.E., 2005. <strong>Management</strong> Research as a Design Science: Articulating the Research Products of Mode<br />

2 Knowledge Producti<strong>on</strong> in <strong>Management</strong>. British Journal of <strong>Management</strong>, 16(1), pp.19-36. Available at:<br />

http://dx.doi.org/10.1111/j.1467-8551.2005.00437.x.<br />

Zarabzadeh, A. et al., 2012. Features of Electr<strong>on</strong>ic Early Warning Systems which Impact Clinical Decisi<strong>on</strong><br />

Making. In The 25th IEEE Internati<strong>on</strong>al Symposium <strong>on</strong> Computer-Based Medical Systems (CBMS 2012).<br />

Rome, Italy.<br />

350


Using the REA Approach to Modeling of IT Process<br />

Evaluati<strong>on</strong><br />

Ryszard Zygala<br />

Wroclaw Universuty of Ec<strong>on</strong>omics, Wroclaw, Poland<br />

ryszard.zygala@ue.wroc.pl<br />

Abstract: For many businesses, Informati<strong>on</strong> Technology (IT) soluti<strong>on</strong>s play a strategic role in gaining and<br />

maintaining competitive advantage. For several decades, a business informati<strong>on</strong> system infrastructure has been<br />

persistently developed and hence it is increasingly more complex. Well organized IT processes have become<br />

more crucial than ever before. IT executives (CIOs) have to make decisi<strong>on</strong>s based <strong>on</strong> high quality informati<strong>on</strong><br />

c<strong>on</strong>cerning various features, how IT processes are planned, managed and improved. In order to successfully<br />

evaluate and manage IT processes, CIOs are supported by dedicated software and hardware soluti<strong>on</strong>s, best<br />

practices and standards. The IT processes evaluati<strong>on</strong> is much more effective when the source data comes from<br />

tailored soluti<strong>on</strong>s for given informati<strong>on</strong> needs. There are a lot of opportunities to develop an informati<strong>on</strong> system<br />

architecture for the IT process evaluati<strong>on</strong> using various software tools. Today, we are facing the important<br />

questi<strong>on</strong> of whether or not it is possible to create software strictly dedicated to an IT management domain, which<br />

also would be an integral part of the ERP architecture and more suitable for SMEs. An attempt to give an answer<br />

to this questi<strong>on</strong> is the main goal of this paper. We propose to use the Resource-Event-Agent (REA) approach to<br />

modeling IT process evaluati<strong>on</strong>. This is an important assumpti<strong>on</strong> because REA lets us see an IT management<br />

domain both as a set of mutually c<strong>on</strong>nected business activities and as a part of the interests of accounting<br />

records. Therefore, the REA modeling makes it possible to describe the IT realm to satisfy informati<strong>on</strong> needs, for<br />

both accountants and n<strong>on</strong>-accountants.<br />

Keywords: IT process evaluati<strong>on</strong>, REA modeling, ITSM system<br />

1. Introducti<strong>on</strong><br />

Depending <strong>on</strong> business size and type, it is estimated that IT-based expenditure may exceed<br />

50 percent of total company capital investments (Posthumusa, v<strong>on</strong> Solms 2005). The current<br />

ec<strong>on</strong>omic recessi<strong>on</strong> is more c<strong>on</strong>ducive to the improvement of business processes, workforce<br />

effectiveness and cutting IT costs (Gruman 2010). Al<strong>on</strong>g with this level of expansi<strong>on</strong>, IT managers are<br />

facing great challenges to meet the business expectati<strong>on</strong>s to provide high quality informati<strong>on</strong> that<br />

generates adequate value and ensures effectiveness of all business processes. This state of<br />

expectati<strong>on</strong>s has led to a spate of discussi<strong>on</strong> c<strong>on</strong>cerning the return <strong>on</strong> IT investment and how to<br />

c<strong>on</strong>trol IT activities. Generally, CIOs have already perceived IT management as an important<br />

technological problem (Gruman 2010), and the IT process evaluati<strong>on</strong> and c<strong>on</strong>trol usually have been<br />

performed manually for last decades, but today IT management tasks are often automated (Dubie<br />

2008). Usually, each of main business functi<strong>on</strong>s (i.e. producti<strong>on</strong>, sales, accounting etc.) is<br />

implemented as a part of an integrated software package. Today, the informati<strong>on</strong> technology<br />

management is undoubtedly very important for numerous businesses, but the software supporting this<br />

area is functi<strong>on</strong>ally incomplete, heterogenous and often isolated from the main enterprise system<br />

architecture. Some leading ERP and business intelligence providers do offer their software tools for IT<br />

management and evaluati<strong>on</strong> support, but they have some disadvantages, i.e.:<br />

There are multi-module software packages which have to be integrated by using additi<strong>on</strong>al<br />

middleware layer (e.g. Oracle),<br />

Leading ERP soluti<strong>on</strong>s often are “outsized” for small and medium enterprises (SME),<br />

There is business intelligence (BI) software specialized in IT process evaluati<strong>on</strong> and management<br />

(e.g. from SAS), but generally BI is not useful without a suitable operati<strong>on</strong>al database,<br />

Existing soluti<strong>on</strong>s are assembled from various comp<strong>on</strong>ents of different origin (e.g. Oracle).<br />

Today, we face the important questi<strong>on</strong> of whether or not it is possible to create a software strictly<br />

dedicated to an IT management domain, which also would be an integral part of the ERP architecture<br />

and more suitable for SMEs. An attempt to give an answer to this questi<strong>on</strong> is the main goal of this<br />

paper. The main scope of the current discussi<strong>on</strong> about IT process management is oriented towards<br />

ITIL and Cobit - the widely known best practices (de facto standards). These standards are important<br />

not <strong>on</strong>ly for the IT management practice, but also for designing software which would support and<br />

automate business processes in this domain. Currently, a rich offer of IT Service <strong>Management</strong> (ITSM)<br />

software can be observed. The existing models of ITSM seem not to be suitable for enterprises with<br />

limited IT resources, both material and human. Small and medium enterprises are those which have<br />

351


Ryszard Zygala<br />

insufficient capital for IT investment and besides, they often have insufficient knowledge to assess<br />

advantages and potential threats related to offered software. We propose to use the Resource-Event-<br />

Agent (REA) approach to modeling IT process evaluati<strong>on</strong>. This is an important assumpti<strong>on</strong> because<br />

REA lets us see an IT management domain both as a set of mutually c<strong>on</strong>nected business activities<br />

and as a part of the interests of accounting records. Therefore, the REA modeling makes possible to<br />

describe the IT realm to satisfy informati<strong>on</strong> needs, for both accountants and n<strong>on</strong>-accountants.<br />

2. The scope of IT process evaluati<strong>on</strong><br />

In the last two decades, and especially since the beginning of the millennium, IT investments payoff<br />

has come to play an increasingly greater part in the debate <strong>on</strong> informati<strong>on</strong> and IT management.<br />

However, informati<strong>on</strong> technology investment has no direct value itself, but it “has potential for derived<br />

value” (Remenyi et al 2000). The essence of what computers produce is in the form of strings of bits,<br />

which finally, as processing outputs, meet various informati<strong>on</strong> needs, direct and indirect informati<strong>on</strong><br />

systems users. These outputs are intangible by nature and they are used in form of output data,<br />

screen c<strong>on</strong>tent, database views, multimedia files, internet portals, communicati<strong>on</strong> channel, sales<br />

channel, virtualized business process and so <strong>on</strong>. Each of the menti<strong>on</strong>ed outputs has a value, which is<br />

often different for different people, so they will further be called informati<strong>on</strong> goods, using informati<strong>on</strong><br />

ec<strong>on</strong>omics terminology. According to Shapiro and Varian, informati<strong>on</strong> goods are type of products “that<br />

essentially c<strong>on</strong>sist of informati<strong>on</strong>, where informati<strong>on</strong> refers to anything that can be digitized, that is,<br />

represented as a streams of bits” (Shapiro, Varian 1998, p. 3). Kaplan and Nort<strong>on</strong> argue that<br />

intangible assets “are worth far more to many companies than their tangible assets”, because they<br />

“are hard for competitors to imitate” (Kaplan and Nort<strong>on</strong> 2004) but intangible outputs of IT investments<br />

often affect financial performance indirectly.<br />

The main subject of interest in informati<strong>on</strong> management evaluati<strong>on</strong> is c<strong>on</strong>cerned with the life cycles of<br />

computerized informati<strong>on</strong> systems. By c<strong>on</strong>ducting literature review and observing business practice it<br />

is possible to draw some interesting c<strong>on</strong>clusi<strong>on</strong>s (see: Banister and Remenyi 2000; GAO 1998; Lech<br />

2007; Remenyi et al 2000; Willcocks and Graeser 2001; Zygala, 2009):<br />

Informati<strong>on</strong> technology investment is outside the mainstream of the general business<br />

performance management. IT investment ec<strong>on</strong>omics is first of all a subject of interest of IT<br />

departments, while the sec<strong>on</strong>d area is managed by financial departments. To build effective IT<br />

process evaluati<strong>on</strong> system it is essential to integrate these two areas.<br />

To identify benefits from IT processes it is enough to use general business performance<br />

measures. To express these benefits in financial measures it is often necessary to build an effect<br />

chain to decompose quantity and quality metrics into financial <strong>on</strong>es.<br />

Managers, especially in small and medium enterprises, do not understand “the foreign language”<br />

of complicated formulas which are used in IT projects evaluati<strong>on</strong>. The evaluati<strong>on</strong> of informati<strong>on</strong><br />

systems demands more “user friendly” methods.<br />

The calculati<strong>on</strong> of IT processes costs is limited to the IT departments borders, passing the fact<br />

over that IT costs are being spent not <strong>on</strong>ly within the c<strong>on</strong>fines of IT functi<strong>on</strong>.<br />

Traditi<strong>on</strong>al business cost systems do not facilitate calculating total cost of ownership (TCO) for<br />

informati<strong>on</strong> systems. More valuable for that is the activity based costing (ABC) model.<br />

Tax c<strong>on</strong>siderati<strong>on</strong>s (in Poland) force businesses to classify some IT costs to incorrect periods (i.e.<br />

to minimize business profit).<br />

Businesses avoid allocating indirect IT costs and c<strong>on</strong>centrate <strong>on</strong> allocating direct IT costs.<br />

Major IT projects have not dedicated them costs accounting dimensi<strong>on</strong> entries.<br />

Financial measures, product quality, customer relati<strong>on</strong>s, and customer satisfacti<strong>on</strong> and faster<br />

delivery times dominate in IT accounting.<br />

Apart from IT investment evaluati<strong>on</strong>, organizati<strong>on</strong>s pay increasingly more attenti<strong>on</strong> to their IT process<br />

efficiency and effectiveness. There are some major advantages of identifying an organizati<strong>on</strong> from<br />

business process perspective. First, it may be possible to answer the questi<strong>on</strong> “how the work is<br />

d<strong>on</strong>e?” Sec<strong>on</strong>d, the important characteristics are defined in every business process, i.e. the beginning<br />

and end of process, the order of activities in time and space, inputs and outputs. Third, the business<br />

process approach implicates a customer point of view. Finally, the business process is a structure,<br />

which generates a business value for customers (see Davenport 1993). The above-menti<strong>on</strong>ed<br />

characteristics can be identified for every IT business process.<br />

352


Ryszard Zygala<br />

Today, it is comm<strong>on</strong>ly accepted that IT business-units work for internal and external customers to<br />

identify and meet their needs. This results in change in the IT management approach: from<br />

technology-oriented to service-oriented (Keel et al 2007), therefore IT service management (ITSM)<br />

has gained popularity in various organizati<strong>on</strong>s. In this paper we recognize ITSM as a crucial part of IT<br />

process management. This distincti<strong>on</strong> is particularly important for developing an IT process evaluati<strong>on</strong><br />

model. According to ITIL Glossary the term service is defined as "A means of delivering value to<br />

customers by facilitating outcomes customers want to achieve without the ownership of specific costs<br />

and risks" (ITIL Glossary) and based <strong>on</strong> ISACA process is "a collecti<strong>on</strong> of activities that takes <strong>on</strong>e or<br />

more kinds of input and creates an output that is of value to the organizati<strong>on</strong>" (Cobit 5). These<br />

definiti<strong>on</strong>s represent two widely known best practices (standards de facto) for the IT management<br />

domain, i.e. Informati<strong>on</strong> Technology Infrastructure Library (ITIL) and C<strong>on</strong>trol Objectives for<br />

Informati<strong>on</strong> and related Technology (Cobit). From the ec<strong>on</strong>omic point of view, services are intangible<br />

products (Waters 1996). Thus, IT services can be recognized as intangible outputs of IT processes,<br />

therefore Cobit and ITIL are more complementary than competitive standards. IT process evaluati<strong>on</strong><br />

is <strong>on</strong>e of the most important objectives in both frameworks. The ITIL framework is more detailed and<br />

operati<strong>on</strong>al than COBIT, and widely accepted approach to IT service management in the world, and<br />

therefore it is often taken to be a subject of computer implementati<strong>on</strong>. COBIT appears to be the best<br />

framework to IT process c<strong>on</strong>trol and evaluati<strong>on</strong> at all levels of IT management. COBIT in versi<strong>on</strong> 4.1<br />

supports IT Governance, and its IT process evaluati<strong>on</strong> features are in line with the c<strong>on</strong>cept of Robert<br />

Kaplan and David Nort<strong>on</strong>’s balanced scorecard. COBIT covers 34 IT processes, grouped into four IT<br />

management domains: Plan and Organize, Acquisiti<strong>on</strong> and Implementati<strong>on</strong>, Delivery and Support,<br />

M<strong>on</strong>itoring and Evaluati<strong>on</strong>. Each of these processes is measured from three perspectives:<br />

1. Organizati<strong>on</strong>al maturity. Each of the COBIT processes has defined six maturity levels, which allow<br />

benchmarking and identificati<strong>on</strong> of the organizati<strong>on</strong>al weaknesses.<br />

2. IT process effectiveness. Each of the COBIT processes has defined outcome measures - lag<br />

indicators, which are used to mean how the goals are met.<br />

3. IT process performance. Each of the COBIT processes has defined performance measures – lead<br />

indicators, which are used to assess process performance.<br />

The evaluati<strong>on</strong> of IT costs, benefits and risks is included in both above described frameworks, and<br />

thus they can be implemented as a sufficient source of data for evaluati<strong>on</strong> of IT processes in SMEs.<br />

Typical functi<strong>on</strong>ality in an ITSM system encompasses all major areas of ITIL best practices in an IT<br />

service <strong>Management</strong> envir<strong>on</strong>ment, such as: availability management, change management, security<br />

management, incident management and others. There is general c<strong>on</strong>sensus in literature that<br />

informati<strong>on</strong> systems are based <strong>on</strong> algorithms; therefore it is easier to develop IT soluti<strong>on</strong>s in wellstructured<br />

and standardized business domains (e.g. Accounting, Payroll). Standardizati<strong>on</strong> of IT<br />

management area seems to be mature enough to design informati<strong>on</strong> systems supporting this<br />

business functi<strong>on</strong>. Johns<strong>on</strong> et al (2007) proposes to develop IT Service <strong>Management</strong> soluti<strong>on</strong>s that<br />

are based <strong>on</strong> publicly available and implementable standards that cover three areas:<br />

IT processes - ITIL, ISO 20000,<br />

data/metadata - CMDB (c<strong>on</strong>figurati<strong>on</strong> management database), Service Modeling Language<br />

(SML), and Soluti<strong>on</strong> Deployment Descriptor (SDD),<br />

and management protocols - standards related to Web Services Distributed <strong>Management</strong><br />

(WSDM).<br />

Generally, this propositi<strong>on</strong> is in line with c<strong>on</strong>temporary technology and practice in ITSM soluti<strong>on</strong>s,<br />

however it does not make allowances for important standards and good practices for IT processes<br />

evaluati<strong>on</strong>, i.e. Cobit, Capability Maturity Models, IT Scorecard, TCO, PMI / PMBOK, ISO standards<br />

for software quality, etc. Additi<strong>on</strong>ally, the accounting standards are essential in order to effectively<br />

integrate an ITSM architecture with an accounting informati<strong>on</strong> system. Today’s small and medium<br />

businesses appreciate the advantages of implementing best practices in Service <strong>Management</strong>. As<br />

was menti<strong>on</strong>ed above, the leading ITSM software providers mainly focus <strong>on</strong> manufacturing ITSM<br />

systems for large businesses based <strong>on</strong> the ITIL standard. These soluti<strong>on</strong>s are not adjusted due to the<br />

cost of purchase and implementati<strong>on</strong>, functi<strong>on</strong>ality, and various maintenance requirements. Desirable<br />

features for an IT process evaluati<strong>on</strong> system may include:<br />

353


Ryszard Zygala<br />

The ability to set up and c<strong>on</strong>figure an IT process repository where each process would be<br />

integrated within a general enterprise system architecture, and first of all with an accounting<br />

functi<strong>on</strong>.<br />

the ability to define and exploit a set of key performance indicators to enable c<strong>on</strong>tinuous process<br />

improvement,<br />

the capability to support both operati<strong>on</strong>al and investment tasks,<br />

The capability to support the open standards for IT service and process management, especially<br />

ITIL, Cobit and project management.<br />

3. The REA model<br />

The Resource-Event-Agent (REA) model was originally designed by McCarthy (McCarthy 1982) as an<br />

alternative for traditi<strong>on</strong>ally designed accounting informati<strong>on</strong> systems. The main goal of his work was<br />

to propose a new model of accounting, where “both accountants and n<strong>on</strong>-accountants are interested<br />

in maintaining informati<strong>on</strong> about the same set of phenomena” (McCarthy 1982). It is important to<br />

emphasize that McCarthy’s model arose under the str<strong>on</strong>g influence of Sorter’s “events” accounting<br />

theory (see McCarthy 1979), and further he perceived the REA Accounting Informati<strong>on</strong> System (AIS)<br />

as a part of “events accounting” (Dunn and McCarthy 1997). The REA model identifies accounting<br />

transacti<strong>on</strong>s from the point of view of such primitives as resource, event, agent, stock-flow,<br />

participati<strong>on</strong> and duality. The generalized REA framework is shown in Figure 1.<br />

Ec<strong>on</strong>omic<br />

Resource<br />

types of<br />

entities<br />

stock<br />

flow<br />

types of<br />

associati<strong>on</strong>s<br />

duality<br />

Ec<strong>on</strong>omic<br />

Event<br />

resp<strong>on</strong>sibility<br />

Ec<strong>on</strong>omic<br />

Agent<br />

c<strong>on</strong>trol<br />

Ec<strong>on</strong>omic<br />

Unit<br />

Figure 1: REA framework. Adaptati<strong>on</strong> based <strong>on</strong> (McCarthy 1982).<br />

Primitives used in this model were defined based <strong>on</strong> the accounting and ec<strong>on</strong>omic theory. Thus,<br />

ec<strong>on</strong>omic resources are the things that can be used (ec<strong>on</strong>omic event) by an ec<strong>on</strong>omic agent.<br />

Ec<strong>on</strong>omic events are defined as activities that change the state of the enterprise (ec<strong>on</strong>omic)<br />

resources. Agents are individuals or organizati<strong>on</strong> units that manage or participate in ec<strong>on</strong>omic events.<br />

Stock-flows are relati<strong>on</strong>ships (associati<strong>on</strong>s) between ec<strong>on</strong>omic events and ec<strong>on</strong>omic resources that<br />

increment or decrement the stock of resources. Duality associati<strong>on</strong>s c<strong>on</strong>nect each increment event<br />

with related to their decrement events. (McCarthy 1982). In formalizing his model, McCarthy used the<br />

entity-relati<strong>on</strong>ship (ER) method of database modeling in order to design a semantic schema of<br />

enterprise business processes. The choice of this method was significant for him because a semantic<br />

model of REA enterprise is not technology dependent and it may be used both as a tool of<br />

communicati<strong>on</strong> during requirements analysis, and as an element of the database system engineering<br />

(Geerts 2008). In the initial stage of its development the REA model was limited to modeling and<br />

analysis of the accounting realm. The types of entities and associati<strong>on</strong>s used in the early stage of the<br />

REA model development did not go bey<strong>on</strong>d the borders of the widely accepted accounting theory.<br />

Further research has significantly extended this model and more and more researchers<br />

have dedicated their endeavors to change its objectives, scope and capability to formalize a subject of<br />

analysis and design. In the 1990s, the REA model was an area of applied research, e.g. Grabski and<br />

Marsh (1994) presented the method of implementati<strong>on</strong> of activity-based costing (ABC) using a REAbased<br />

system, and Denna, Jaspers<strong>on</strong>, F<strong>on</strong>g, and Middleman (1994) dem<strong>on</strong>strated the modeling of<br />

354


Ryszard Zygala<br />

c<strong>on</strong>versi<strong>on</strong> processes within an REA system. Next, Walker and Denna (1997) gave several examples<br />

of acceptance of the event-driven technology by various business and public organizati<strong>on</strong>s. In their<br />

extensive textbook, Denna, Cherringt<strong>on</strong> and Hollander (1995) proposed the REAL approach to design<br />

AIS with added locati<strong>on</strong> dimensi<strong>on</strong>. Dunn and McCarthy (1997) postulate developing REA research<br />

both as a part of design science, related to computer science and as a part of natural science.<br />

As menti<strong>on</strong>ed earlier, the REA model was originally developed as an alternative way of describing<br />

enterprise activities from the accounting perspective. Geerts and McCarthy have extended the scope<br />

of the REA model from the accounting domain to the enterprise domain <strong>on</strong>tology. They proposed<br />

some new REA primitives, viewed both from business process granularity (i.e. entrepreneur script,<br />

process, and task) and as “type images for all these phenomena as well as relati<strong>on</strong>ships between<br />

these type images” (Geerts and McCarthy 2000). They follow Gruber’s definiti<strong>on</strong> of <strong>on</strong>tology as “an<br />

explicit specificati<strong>on</strong> of c<strong>on</strong>ceptualizati<strong>on</strong>” (Gruber 1993). These researchers have further analyzed<br />

REA <strong>on</strong>tology based <strong>on</strong> the c<strong>on</strong>ceptual terminology of John Sowa (Geerts McCarthy, 2002). David et<br />

al (2002) examined the current state of the REA design science research and, am<strong>on</strong>g other things,<br />

they suggest including intangible assets as resources in the REA modeling practice.<br />

Geerts and McCarthy (2006) used abstracti<strong>on</strong>s to specify policy-level extensi<strong>on</strong>s to REA enterprise<br />

systems, where the term ”policy” means “a descripti<strong>on</strong> of ec<strong>on</strong>omic phenomena that could, should, or<br />

must occur”. The REA model was developed using entity-relati<strong>on</strong>ship diagramming techniques;<br />

nevertheless, it may be practiced with other modeling tools, esp. Unified Modeling Language (UML).<br />

We propose to use the Resource-Event-Agent (REA) approach to modeling IT processes evaluati<strong>on</strong>.<br />

There are several key advantages of this approach. Firstly, the REA approach is especially useful for<br />

modeling business processes according to the accounting principles. REA modeling <strong>on</strong> its semantic<br />

level rejects the traditi<strong>on</strong>ally used accounting artifacts, such as debits, credits, and accounts. These<br />

artifacts can be delivered to users as database views. Sec<strong>on</strong>dly, the REA c<strong>on</strong>cept was used as a<br />

theoretical background for the ISO standard: ISO/IEC 15944 c<strong>on</strong>cerning accounting and ec<strong>on</strong>omic<br />

<strong>on</strong>tology. Thirdly, the REA framework has evolved into business <strong>on</strong>tology, therefore it may include<br />

and integrate both accounting and n<strong>on</strong>-accounting business processes.<br />

4. Modeling IT process evaluati<strong>on</strong> based <strong>on</strong> REA approach<br />

In this secti<strong>on</strong> of the paper we propose some patterns that allow modeling IT process evaluati<strong>on</strong> using<br />

REA principles. There are many business producers of software that offer ITIL support, but their<br />

software tools for IT process evaluati<strong>on</strong> have some disadvantages, i.e.:<br />

There are multi-module software packages which have to be integrated by using an additi<strong>on</strong>al<br />

middleware layer,<br />

There is business intelligence (BI) software specialized in IT process evaluati<strong>on</strong> and<br />

management, but generally BI isn’t useful without a suitable operati<strong>on</strong>al database,<br />

Existing soluti<strong>on</strong>s are assembled from various comp<strong>on</strong>ents of different origin,<br />

Leading ERP soluti<strong>on</strong>s often are “outsized” for small and medium enterprises (SME).<br />

Given the above, the fundamental questi<strong>on</strong> seems to be how to successfully build an optimal<br />

informati<strong>on</strong> system for SMEs, covering the most important aspects of the IT process evaluati<strong>on</strong> area.<br />

In order to answer this questi<strong>on</strong>, it is important to take into c<strong>on</strong>siderati<strong>on</strong> that IT adopti<strong>on</strong> and<br />

management in above-menti<strong>on</strong>ed enterprises are different than in larger firms. First, SMEs usually<br />

employ multi-skilled workers with the lack of str<strong>on</strong>g IT/IS knowledge and technical skills. From this it<br />

follows that software for IT managers in SMEs should be adjusted to both their scope of resp<strong>on</strong>sibility<br />

and to their time budget. Sec<strong>on</strong>d, the informati<strong>on</strong> system users often have to be more independent in<br />

their own daily routines. Third, SMEs usually have limited financial resources to IT investments, hence<br />

software supporting IT management tasks should be characterized by simplicity, "all in <strong>on</strong>e"<br />

functi<strong>on</strong>ality and low costs of purchase and maintenance.<br />

As was earlier menti<strong>on</strong>ed, the c<strong>on</strong>temporary service approach has become dominant in IT<br />

management issues. During the <strong>on</strong>going discussi<strong>on</strong> about REA-based areas of business soluti<strong>on</strong>s,<br />

research is focusing <strong>on</strong> the processes of the physical value chain where inputs and outputs are<br />

tangible and customers are external. IT processes are a special type of operati<strong>on</strong>al business activity<br />

that often have their inputs and outputs intangible and their customers are internal users of different<br />

informati<strong>on</strong> systems. The main REA research focuses <strong>on</strong> the crucial ec<strong>on</strong>omic events c<strong>on</strong>cerning<br />

355


Ryszard Zygala<br />

major business processes: sales, procurement, and manufacturing. In more complex organizati<strong>on</strong>s an<br />

informati<strong>on</strong> system architecture includes data about many internal business activities. Therefore, for<br />

our purposes, especially useful may be the extensi<strong>on</strong> proposed by David (David 1997), where the<br />

three types of events are defined:<br />

1) Ec<strong>on</strong>omic events - "events that increase or decrease the quantity of a firm’s resources";<br />

2) Business events - "any business activity that management wants to plan, m<strong>on</strong>itor, and evaluate";<br />

3) Informati<strong>on</strong> events - "procedures that are performed in organizati<strong>on</strong>s solely to capture, manipulate,<br />

or communicate informati<strong>on</strong>".<br />

When comparing this propositi<strong>on</strong> to the basic REA pattern, it is important to emphasize that business<br />

events comprise "a subset of the tasks" in the Geerts and McCarthy model and "they do not<br />

participate in duality relati<strong>on</strong>ships" and hence they are support activities in the business value chain<br />

(David et al 2002). The three types of events are important in the IT process evaluati<strong>on</strong> domain and<br />

should be designed, especially when the event type may be used as a special feature which<br />

delimitates the accounting and n<strong>on</strong>-accounting realm. The sec<strong>on</strong>d extensi<strong>on</strong> to the core REA<br />

terminology c<strong>on</strong>cerns the definiti<strong>on</strong> of ec<strong>on</strong>omic resource. In his research McCarthy used the term<br />

resource in the accounting c<strong>on</strong>text, and therefore we propose to use the definiti<strong>on</strong> proposed by<br />

Romney and Steinbart (2006). They define resources as things that have ec<strong>on</strong>omic value to the<br />

organizati<strong>on</strong>, hence this definiti<strong>on</strong> embraces both tangible and end intangible resources. In order to<br />

present the usefulness of the REA approach for modeling IT process evaluati<strong>on</strong>, c<strong>on</strong>sider for example<br />

IT cost allocati<strong>on</strong>.<br />

Several researchers (am<strong>on</strong>g others (Grabski, Marsh 1994), (David et al, 2002) have discussed a<br />

relati<strong>on</strong>ship between the ABC costing model and REA. It is noticeable that there are c<strong>on</strong>siderable<br />

difficulties to embed the ABC model within the organizati<strong>on</strong>-wide informati<strong>on</strong> system architecture.<br />

However, we believe that these difficulties can be overcome by changing accounting informati<strong>on</strong><br />

system architectures from account-centric to process-centric. In order to achieve such a purpose, the<br />

REA approach seems to be effective in practice.<br />

If, having regard to the nature of the work executed by IT specialists, the time-driven activity-based<br />

costing (TDABC) model appears to be more adequate than the traditi<strong>on</strong>al model. What is more,<br />

according to (Kaplan and Nort<strong>on</strong> 2004), TDABC "is simpler, less costly, and faster to implement, and<br />

allows cost driver rates to be based <strong>on</strong> the practical capacity of the resources supplied". They argue<br />

that <strong>on</strong>ly two parameters have to be estimated in this model: "(1) the unit cost of supplying capacity<br />

and (2) the time required to perform a transacti<strong>on</strong> or an activity".<br />

Designing an IT process evaluati<strong>on</strong> system requires integrating traditi<strong>on</strong>al accounting data, n<strong>on</strong>traditi<strong>on</strong>al<br />

accounting data (from TDABC) and IT management data. It can be called a<br />

multidimensi<strong>on</strong>al approach, where an individual IT process can be evaluated in terms of<br />

various indicators of performance, quality, and efficiency.<br />

We assume that the TDABC procedure begins by estimating the cost of IT department capacity<br />

(Figure 2 and 3). C<strong>on</strong>sider a centralized IT department (ITD) as a cost center, thus it means that it is<br />

not directly resp<strong>on</strong>sible for the business profit. To simplify further steps, we assume that all IT costs<br />

are incurred and accounted for in ITD. The IT manager has the authority to incur different direct and<br />

indirect costs related to the services provided by ITD for internal users. Based <strong>on</strong> (Remenyi et al.<br />

2000), the IT costs portfolio encompasses direct costs of hardware, software, services, overheads,<br />

training, and maintenance, as well as indirect human and organizati<strong>on</strong>al costs. Having discussed the<br />

nature of service, we have c<strong>on</strong>cluded that a service is an intangible product, and therefore, in the<br />

REA logic, an IT service is an important type of IT resources. It can be useful to perceive the IT<br />

domain as an ec<strong>on</strong>omic system in which different inputs (Resources) are c<strong>on</strong>verted (Events) into<br />

valuable outputs (Resources). The c<strong>on</strong>versi<strong>on</strong> events are performed by internal IT specialists<br />

(Agents) to satisfy different informati<strong>on</strong> needs of informati<strong>on</strong> system users (Agents) (see Figure 2).<br />

356


Ryszard Zygala<br />

Figure 2: Exchange process of ec<strong>on</strong>omic events in ITD<br />

The term ec<strong>on</strong>omic event in the REA <strong>on</strong>tology means either increment or decrement in the value of<br />

ec<strong>on</strong>omic resources, hence ITD can do that by either exchange or c<strong>on</strong>versi<strong>on</strong> process. The exchange<br />

process usually encompasses relati<strong>on</strong>s (interacti<strong>on</strong>s) between ITD and its external partners (e.g.<br />

software providers). Every increment ec<strong>on</strong>omic event is related to a decrement ec<strong>on</strong>omic event<br />

(exchange duality). Inflow relati<strong>on</strong>ships c<strong>on</strong>nect increment ec<strong>on</strong>omic events with ec<strong>on</strong>omic resources.<br />

Every ec<strong>on</strong>omic event is related to an ec<strong>on</strong>omic agent, using "provide" or "receive" relati<strong>on</strong>ship. The<br />

developed set of REA principles enables to design the business logic according to the accounting<br />

standards. The accounting logic is often not sufficient for internal purposes and therefore it is so<br />

important to enlarge the scope of REA <strong>on</strong>tology of business and informati<strong>on</strong> events.<br />

<br />

Hardware<br />

Cost (Value)<br />

<br />

Service<br />

Cost (Value)<br />

<br />

Labor<br />

Cost (Value)<br />

<br />

(or)<br />

<br />

<br />

<br />

<br />

Allocate Cost<br />

of Fixed Asets<br />

<br />

Allocate Cost<br />

of Current Assets<br />

<br />

Allocate Cost<br />

of Services<br />

<br />

Allocate Cost<br />

of Labor<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

<br />

Fixed Assets<br />

System<br />

<br />

Purchasing<br />

System<br />

<br />

Payroll<br />

System<br />

<br />

<br />

Cost <strong>Management</strong><br />

System<br />

<br />

IT <strong>Management</strong><br />

System<br />

<br />

<br />

Depreciati<strong>on</strong> File<br />

Cost (Value)<br />

<br />

Purchasing File<br />

Cost (Value)<br />

<br />

Payroll File<br />

Cost (Value)<br />

Set of tools to<br />

allocate and<br />

manage costs<br />

<br />

ITD Costs File<br />

Priod (Date)<br />

Cost (Value)<br />

Figure 3: The REA design for informati<strong>on</strong> events<br />

In this paper, we accept the extensi<strong>on</strong> of the REA <strong>on</strong>tology, proposed by David (David 1997) to the<br />

informati<strong>on</strong> event noti<strong>on</strong>. The informati<strong>on</strong> events in the picture 3 represent the cost allocati<strong>on</strong><br />

357


Ryszard Zygala<br />

procedures that "solely to capture, manipulate, or communicate informati<strong>on</strong>" (see above). The<br />

informati<strong>on</strong> events are comm<strong>on</strong>ly identified in business practice, but it may be c<strong>on</strong>troversial to include<br />

them in the REA <strong>on</strong>tology <strong>on</strong> the same level of an events hierarchy as ec<strong>on</strong>omic and business<br />

events. It seems that informati<strong>on</strong> events are rather a subset of ec<strong>on</strong>omic and business events. As is<br />

shown in the picture 3, we propose to perceive an informati<strong>on</strong> system entity as an informati<strong>on</strong> agent<br />

in REA terminology because of their participati<strong>on</strong> in informati<strong>on</strong> event. Generally, the REA <strong>on</strong>tology<br />

does not descend to a data flow level but in the network ec<strong>on</strong>omy a data flow level more often have<br />

become de facto a business process level. To complete the modeling process in given domain, it is<br />

necessary to descend to the lowest level, where all types of entities are digital (see Picture 4).<br />

Modeling the functi<strong>on</strong>ality of the IT process evaluati<strong>on</strong>, it seems to be crucial to reflect a usage scale<br />

of informati<strong>on</strong> entities, such as informati<strong>on</strong> events, informati<strong>on</strong> resources and informati<strong>on</strong> agents. For<br />

example, whereas a key performance indicators (KPI) are usually implemented as a properties of<br />

given entity, it is more effective to model KPIs as a informati<strong>on</strong> resources. In c<strong>on</strong>sequence of this<br />

change, the whole designed system gains new capabilities in data collecti<strong>on</strong>, processing and<br />

analysis.<br />

Figure 4: The REA design <strong>on</strong> data flow level<br />

All pictures in this point are presented more as a dem<strong>on</strong>strati<strong>on</strong> than implementati<strong>on</strong> diagrams,<br />

thereby presented entities are incomplete and/or not applicable. In practice, the REA approach can be<br />

used in different modeling language, esp. in UML and ERD. Recently, it is noticeable in the REA<br />

literature that UML have gained more followers.<br />

358


Ryszard Zygala<br />

5. C<strong>on</strong>clusi<strong>on</strong> and future research directi<strong>on</strong>s<br />

Managing today's IT infrastructure and processes have become increasingly complex. IT managers<br />

face the problem of meeting the growing requirements for a positive impact of IT soluti<strong>on</strong>s <strong>on</strong> the<br />

business effectiveness of both operati<strong>on</strong>al and investment activities. To meet these requirements, the<br />

high quality of informati<strong>on</strong> about IT processes and services must be ensured. For that reas<strong>on</strong>, it is so<br />

important to develop informati<strong>on</strong> system soluti<strong>on</strong>s which would be a sufficient source of data for the IT<br />

process evaluati<strong>on</strong>.<br />

The REA <strong>on</strong>tology encompasses different characteristics that are important from the practical point of<br />

view. First of all, it enables to perceive the business realm as a set of related events which can be<br />

described and evaluated from different perspectives. In design practice the term “event” can be used<br />

interchangeably as an equivalent of “activity”, “process” or “task”, despite their distincti<strong>on</strong> in the REA<br />

<strong>on</strong>tology literature. The nature of the REA approach to the informati<strong>on</strong> system design is the modeling<br />

of business domain by a well-defined set of entity and associati<strong>on</strong>s types that have gained the<br />

acceptance of the ec<strong>on</strong>omic and accounting theory. There are various informati<strong>on</strong> needs c<strong>on</strong>cerning<br />

each individual ec<strong>on</strong>omic event, and financial accounting is <strong>on</strong>e of the many perspectives that do<br />

matter for IT management practice.<br />

As emphasized earlier, modeling IT process management soluti<strong>on</strong>s should take into account publicly<br />

available and implementable standards that c<strong>on</strong>cern: IT processes, IT data and tools, management<br />

protocols, and IT process evaluati<strong>on</strong> (e.g. ITIL, Cobit, Capability Maturity Models, IT Scorecard, TCO).<br />

We recommend for future research to develop models of IT management systems, where process<br />

evaluati<strong>on</strong> will be fully involved. It is also hoped that this paper will trigger a debate about designing IT<br />

process evaluati<strong>on</strong>, and it is our belief that the REA approach may be helpful.<br />

References<br />

Bannister, F. and Remenyi, D. (2000) "Acts of faith: instinct, value and IT investment decisi<strong>on</strong>s", Journal of<br />

Informati<strong>on</strong> Technology, 2000/15, pp. 231-241.<br />

Davenport, T. (1993) Process Innovati<strong>on</strong>: Reengineering Work through Informati<strong>on</strong> Technology, Harvard<br />

Business School Press, Bost<strong>on</strong> 1993.<br />

David, J.S. (1997) "Three "Events" That Define an REA Approach to Systems Analysis, Design, and<br />

Implementati<strong>on</strong>". In Proceedings of the Annual Meeting of the American Accounting Associati<strong>on</strong>, Dallas, TX.<br />

David, J.S., Gerard, G.J., and McCarthy, W.E. (2002) "Design science: An REA perspective <strong>on</strong> the future of AIS",<br />

in Researching Accounting as an Informati<strong>on</strong> Systems Discipline, edited by V. Arnold, and S.G. Sutt<strong>on</strong>, 35-<br />

64. Sarasota, FL: American Accounting Associati<strong>on</strong> Informati<strong>on</strong> Systems Secti<strong>on</strong>.<br />

Dubie, D. (2008) "ITIL, automati<strong>on</strong> provide recipe for datacenter management", InfoWorld, April 01, 2008,<br />

[<strong>on</strong>line], https://www.infoworld.com/d/mobilize/itil-automati<strong>on</strong>-provide-recipe-datacenter-management-888.<br />

Dunn, C.L. and McCarthy, W.E. (1997) "The REA accounting model: Intellectual heritage and prospects for<br />

progress", Journal of Informati<strong>on</strong> Systems (Fall): 251-266.<br />

GAO (1998) "Measuring Performance and Dem<strong>on</strong>strating Results of Informati<strong>on</strong> Technology Investments".<br />

Available: http://www.gao.gov/special.pubs/ai98089.pdf.<br />

Geerts, G. (2008) "Introducti<strong>on</strong> to the REA 25th Anniversary Special Secti<strong>on</strong>", Journal of Informati<strong>on</strong> Systems,<br />

Sarasota, Fall 2008. Vol. 22, Iss. 22; pg. 215, 3 pgs.<br />

Geerts, G. and McCarthy W.E. (2000) "The Ontological Foundati<strong>on</strong> of REA Enterprise Informati<strong>on</strong> Systems",<br />

[<strong>on</strong>line], http://www.msu.edu/~mccarth4/Alabama.doc.<br />

Geerts, G. and McCarthy, W.E. (2002) "An <strong>on</strong>tological analysis of the ec<strong>on</strong>omic primitives of the extended REA<br />

enterprise informati<strong>on</strong> architecture", Internati<strong>on</strong>al Journal of Accounting Informati<strong>on</strong> Systems 3 (1): 1-16.<br />

Geerts, G. and McCarthy, W.E. (2006) "Policy-Level Specificati<strong>on</strong>s in REA Enterprise Informati<strong>on</strong> Systems",<br />

Journal of Informati<strong>on</strong> Systems (Fall): 37-63.<br />

Gruber, T. R. (1993) "A translati<strong>on</strong> approach to portable <strong>on</strong>tology specificati<strong>on</strong>s", Knowledge Acquisiti<strong>on</strong> 5: 199-<br />

220.<br />

Gruman, G. (2010) "CIO priorities shift to lightweight services, virtualizati<strong>on</strong>", InfoWorld, January 19, 2010,<br />

[<strong>on</strong>line], http://www.infoworld.com/d/adventures-in-it/2010-cio-priorities-shift-lightweight-servicesvirtualizati<strong>on</strong>-453.<br />

ITGI (IT Governance Institute), "COBIT 4.1 – digital editi<strong>on</strong>", Rolling Meadows, IL 60008 USA, [<strong>on</strong>line],<br />

www.itgi.org.<br />

ITIL Glossary, [<strong>on</strong>line], http://www.itil-officialsite.com/Internati<strong>on</strong>alActivities/ITILGlossaries_2.aspx).<br />

Johns<strong>on</strong>, M.W., Hately, A., Miller B.A. and Orr R. "Evolving standards for IT service management", IMB Systems<br />

Journal; Jul-Sep 2007; 46, 3 pg 583.<br />

Kaplan, R.S., and Anders<strong>on</strong>, S.R., (2004) "Time-Driven Activity Based Costing", Harvard Business Review, 82<br />

(11): pp.131-138.<br />

359


Ryszard Zygala<br />

Keel, A.J., Orr, M.A., Hernandez, R.R., Patrocinio, E.A. and Bouchard, J. (2007) "From a technology-oriented to<br />

a service-oriented approach to IT management", IBM Systems Journal, Vol. 46, No 3.<br />

McCarthy, W.E. (1979) "An Entity-Relati<strong>on</strong>ship View of Accounting Models", The Accounting Review (October):<br />

667-686.<br />

McCarthy, W.E. (1982) "The REA accounting model: A generalized framework for accounting systems in a<br />

shared data envir<strong>on</strong>ment", The Accounting Review (July): 554-578.<br />

McKinsey Global Institute (2002) "How IT enables productivity Growth", [<strong>on</strong>line],<br />

http://www.mckinsey.com/knowledge/mgi/IT/.<br />

Lech, P. (2007) Proposal of a Compact IT Value Assessment Method, The Electr<strong>on</strong>ic Journal of Informati<strong>on</strong><br />

Systems Evaluati<strong>on</strong>, Vol. 10.<br />

Posthumusa, S. and v<strong>on</strong> Solms, R. (2005) "IT oversight: an important functi<strong>on</strong> of corporate governance",<br />

Computer Fraud & Security, vol 2005, issue 6, June 2005, pp: 11-17.<br />

Remenyi, D., M<strong>on</strong>ey, A. and Sherwood-Smith M. (2000) The effective measurement and management of IT costs<br />

and benefits, Butterworth-Heinemann, Oxford.<br />

Romney, M.B. and Steinbart, P.J. (2006) Accounting Informati<strong>on</strong> Systems, 10th editi<strong>on</strong>. New York, NY: Prentice<br />

Hall.<br />

Shapiro, C. and Varian, V. (1998) Informati<strong>on</strong> Rules, Harvard Business School Press, Harvard.<br />

Walker, K. and Denna, E. (1997) "Arrivederci, Pacioli? A new accounting system is emerging", <strong>Management</strong><br />

Accounting. M<strong>on</strong>tvale: Jul 1997. Vol., 79, Iss. 1; pg. 22, 8 pgs.<br />

Waters, D. (1996) Operati<strong>on</strong> <strong>Management</strong>: Producing Goods and Services. (Polish Editi<strong>on</strong>: PWN, Warszawa<br />

2001).<br />

Willcocks, L., Graeser, V. (2001) Delivering IT and e-Business Value, Butterworth Heinemann, Oxford.<br />

Zygala, R. (2009) "Towards An Informati<strong>on</strong> <strong>Management</strong> Evaluati<strong>on</strong> System: Defining Key Assumpti<strong>on</strong>s" [in]<br />

Informati<strong>on</strong> <strong>Management</strong>, edited by B. Kubiak, A. Korowicki, pp. 375-385, Gdansk University Press,<br />

Gdansk.<br />

360


N<strong>on</strong><br />

<strong>Academic</strong><br />

Papers<br />

361


362


A Process Model to Guarantee Informati<strong>on</strong> Quality in<br />

Elective Surgery Informati<strong>on</strong> Systems<br />

Rita Cristóvão and Pedro Gomes<br />

ACSS - Administração Central do Sistema de Saúde, UCGIC - Unidade Central<br />

de Gestão de Inscritos para Cirurgia, Lisboa, Portugal<br />

rcristovao@acss.min-saude.pt<br />

pagomes@acss.min-saude.pt<br />

Abstract: This paper describes the system created by the Central Unit of Nati<strong>on</strong>al Waiting List for Surgery<br />

<strong>Management</strong> (UCGIC) to guarantee the quality of informati<strong>on</strong> extracted from Nati<strong>on</strong>al Health System (NHS)<br />

hospitals about elective surgery, covering the process of extracti<strong>on</strong>, validati<strong>on</strong> and qualificati<strong>on</strong> of data detail and<br />

indicators, to be carried out automatically by the informati<strong>on</strong> system that supports the waiting list for surgery<br />

management c<strong>on</strong>trol – SIGLIC (Informati<strong>on</strong> System to Waiting List for Surgery <strong>Management</strong>). The need to build<br />

an appropriate process model has been growing since 2007. In 2010 the central database of SIGLIC had to<br />

receive data from nearly 164 hospital units, including public and private providers with c<strong>on</strong>venti<strong>on</strong>s in the NHS for<br />

elective surgery, with a volume of nearly 1000GB and an annual increment approximately of 500GB. The data<br />

received was c<strong>on</strong>cerned to 881 different input variables and the volume of informati<strong>on</strong> transacti<strong>on</strong>s was nearly 5<br />

milli<strong>on</strong> per year. Several problems c<strong>on</strong>cerning data quality in the central database started to arise <strong>on</strong>ce the data<br />

extracti<strong>on</strong> and integrati<strong>on</strong> in SIGLIC involved the interface with several different hospital informati<strong>on</strong> systems and<br />

its volume started to increase fast. The soluti<strong>on</strong> found by UCGIC for addressing data quality and integrity<br />

received from hospital units was to build in 2011 a process model with automatic redundant system with different<br />

sources checking permanently data quality and interacting with all stakeholders involved with the same<br />

informati<strong>on</strong> sources. The model includes m<strong>on</strong>thly data extracti<strong>on</strong>s submitted to a qualificati<strong>on</strong> process at the level<br />

of detail and indicators, against the defined standards and homologous variati<strong>on</strong>s, in order to provide accurate<br />

intelligence about nati<strong>on</strong>al elective surgery. The validati<strong>on</strong> process includes a management system of incidents,<br />

communicati<strong>on</strong>s and escalati<strong>on</strong> of problems, which reports the errors/incidents occurred, the communicati<strong>on</strong> to<br />

the stakeholder involved for correcti<strong>on</strong> and the escalati<strong>on</strong> of the problem resoluti<strong>on</strong> if needed. The process<br />

management of data extracti<strong>on</strong> and qualificati<strong>on</strong> is carried out through SIGLIC own screens/forms and reports. It<br />

has a dashboard and a procedure to ensure process c<strong>on</strong>trol. There were also built scorecards with data<br />

aggregated by week. Studies have been c<strong>on</strong>ducted to assess the impacts and outcomes of this new approach.<br />

By this new model, UCGIC was able to assure that SIGLIC informati<strong>on</strong> is reliable and its performance indicators<br />

are correct and reflect the actual care provided to patients, the hospital performance according to care provided,<br />

the accurate evaluati<strong>on</strong> of demand and supply in elective surgery and the necessary funding for the NHS. Also<br />

this automatic process is c<strong>on</strong>siderably less time and resource c<strong>on</strong>suming, by saving nearly 5 days to a process<br />

which took 10 days and by allowing automatic reports of errors, turning its resoluti<strong>on</strong> with the hospital units more<br />

efficient and effective.<br />

Keywords: data quality, qualificati<strong>on</strong> process, data validati<strong>on</strong>, data extracti<strong>on</strong>, business intelligence<br />

1. Introducti<strong>on</strong><br />

This essay is about a model to informati<strong>on</strong> quality of Portuguese Nati<strong>on</strong>al Health System (NHS)<br />

elective surgery data stored in the central database of the <strong>Management</strong> System of the Waiting List for<br />

Surgery (SIGIC), headquartered in the Central Administrati<strong>on</strong> of NHS - ACSS. The need to build an<br />

efficient model came from the difficulties in assuring data quality within acceptable times due to the<br />

fact that most of the tasks were manual and there were lack of resources to perform quickly and<br />

efficiently indicators and reports. In Secti<strong>on</strong> 2, we describe the problem of ensuring data quality with<br />

the past model and its implicati<strong>on</strong>s in the m<strong>on</strong>itoring and report about NHS elective surgery. In<br />

Secti<strong>on</strong> 3, we explain the details of the new model for data quality assurance and its methodology. In<br />

Secti<strong>on</strong> 4, we measure the results achieved with the applicati<strong>on</strong> of the new model. Finally, in Secti<strong>on</strong><br />

5, we discuss a c<strong>on</strong>clusi<strong>on</strong> about the advantages and challenges of the new informati<strong>on</strong> quality model<br />

developed by the central unit of SIGIC (UCGIC), its implicati<strong>on</strong>s as an innovative system to certify<br />

data quality in SIGIC and possible resp<strong>on</strong>ses to best practices in implementing a data quality system.<br />

2. The problem<br />

SIGIC was created by the Portuguese Ministry of Health to manage and promote the care access in<br />

elective surgery within the NHS. Since its implementati<strong>on</strong> in 2004, SIGIC had to collect informati<strong>on</strong> in<br />

elective surgery provided by all NHS hospitals and private hospitals with NHS c<strong>on</strong>venti<strong>on</strong>s or<br />

protocols to the central database in SIGLIC (SIGLIC is the informati<strong>on</strong> system that supports SIGIC<br />

management and m<strong>on</strong>itoring). From 2007, when SIGIC was already matured, the number of private<br />

hospitals that joined NHS c<strong>on</strong>venti<strong>on</strong>s for elective surgery grew very fast, and with it the need to build<br />

363


Rita Cristóvão and Pedro Gomes<br />

multiple interfaces to collect data from all hospitals with the minimum quality standards. In 2010, the<br />

central database of SIGLIC had to receive data from nearly 164 hospital units, including public and<br />

private providers, with a volume of nearly 1000GB and an annual increment of approximately 500GB.<br />

The data received was c<strong>on</strong>cerned to 881 different input variables and the volume of informati<strong>on</strong><br />

transacti<strong>on</strong>s was nearly 5 milli<strong>on</strong> per year. Several problems c<strong>on</strong>cerning data quality in the central<br />

database started to arise <strong>on</strong>ce the data extracti<strong>on</strong> and integrati<strong>on</strong> in SIGLIC involved the interface<br />

with several different local systems in hospitals and its volume started to increase fast. There were<br />

two main problems. One was the fact that most of hospital informati<strong>on</strong> systems weren´t certified by<br />

SIGIC requirements, which means that they weren´t fully adapted to the SIGIC mandatory standards.<br />

The other problem was the lack of an efficient and effective process of informati<strong>on</strong> quality c<strong>on</strong>trol. The<br />

process in place was too time-c<strong>on</strong>suming and had too many manual tasks performed by a small team<br />

in UCGIC. That represented a risk not <strong>on</strong>ly to the quality of informati<strong>on</strong> accessed by SIGLIC users,<br />

but also to the quality of intelligence about nati<strong>on</strong>al about elective surgery and the waiting list for<br />

surgery in NHS officially released by the Ministry of Health, whose informati<strong>on</strong> source is SIGLIC.<br />

The following table shows the data volume and transacti<strong>on</strong>s of SIGLIC in 2010.<br />

Table 1: Figures of SIGLIC operati<strong>on</strong>al database<br />

2010<br />

Nr of hospital units in SIGIC 116<br />

Nr of applicati<strong>on</strong> users 6.611<br />

Nr of applicati<strong>on</strong> entrees 822.429<br />

Nr of working hours in the applicati<strong>on</strong> 520.896<br />

Nr of data transacti<strong>on</strong>s 4.853.236<br />

Nr input variables 881<br />

Total Data Warehouse (DW) Volume 950 GB<br />

Annual growth 450 GB<br />

To solve the problems related to informati<strong>on</strong> quality, in 2011, the UCGIC started to build a new model<br />

to guarantee the integrity and quality of data received from hospital units, by implementing an<br />

automatic system, with redundant items, different sources checking permanently data quality and<br />

interacting with all stakeholders involved in the process, using the same informati<strong>on</strong> sources.<br />

3. Method/ methodology<br />

UCGIC created an innovative model for validati<strong>on</strong> and qualificati<strong>on</strong> of massive data extracted from<br />

various informati<strong>on</strong> systems based <strong>on</strong> the existing internati<strong>on</strong>al standards but adapted to SIGIC<br />

special needs. The ultimate goal is to implement a system of data quality and data governance in<br />

NHS elective surgery, that will allow the overall management of the availability, usability, integrity, and<br />

security of the data employed in SIGIC.<br />

The model covers all processes of informati<strong>on</strong> quality certificati<strong>on</strong>, from the hospital source to the<br />

central database, including the process of extracti<strong>on</strong>, validati<strong>on</strong> and qualificati<strong>on</strong> of data detail and<br />

indicators, to be carried out automatically by the informati<strong>on</strong> system that supports the waiting list for<br />

surgery management c<strong>on</strong>trol - SIGLIC. It also includes automated processes of detecti<strong>on</strong> and<br />

reporting of errors, correcti<strong>on</strong> of errors in data source, c<strong>on</strong>tinuous m<strong>on</strong>itoring of the process, and the<br />

team assigned in each task of the overall process and resp<strong>on</strong>sible for data's accuracy, accessibility,<br />

c<strong>on</strong>sistency, and completeness.<br />

The methodology used by UCGIC was first to build the model of the informati<strong>on</strong> quality system to be<br />

implemented in SIGLIC, by defining the standards to informati<strong>on</strong> quality certificati<strong>on</strong> in elective<br />

surgery. We designed the management processes and guidelines for the extracti<strong>on</strong>, validati<strong>on</strong> and<br />

qualificati<strong>on</strong> of data uploaded from the local operati<strong>on</strong>al systems in hospitals, all compiled in a<br />

manual, including the role of each team in each task of the process. This process was to be<br />

m<strong>on</strong>itored by SIGLIC. The sec<strong>on</strong>d step was to implement the model in the field by developing the<br />

technical and functi<strong>on</strong>al specificati<strong>on</strong>s in the informati<strong>on</strong> system, assigning the teams with elements of<br />

SIGIC, SIGLIC and the hospitals and involving all the stakeholders in the process.<br />

364


3.1 The process model<br />

Rita Cristóvão and Pedro Gomes<br />

The informati<strong>on</strong> quality model can be divided in the following parts:<br />

The model starts with the integrati<strong>on</strong> of hospital data from its operati<strong>on</strong>al systems into the central<br />

database of SIGLIC via interface and it is submitted to the standards validati<strong>on</strong> through RIS – the IT<br />

network supported by the Ministry of Health and that c<strong>on</strong>nects all public health instituti<strong>on</strong>s and care<br />

providers.<br />

After the data is extracted, it undergoes a process of qualificati<strong>on</strong> with 4 levels and in each level the<br />

informati<strong>on</strong> is saved in a different data store. At level 1, the data produced and exchanged in the<br />

interface with hospitals is gathered and kept in the first data store after passing through the first basic<br />

qualificati<strong>on</strong> standards.<br />

At level 2, the informati<strong>on</strong> is processed in order to be integrated in the operati<strong>on</strong>al database of<br />

SIGLIC, after being approved by the sec<strong>on</strong>d qualificati<strong>on</strong> stage. This procedure has 2 steps; first, it<br />

checks the compliance of data and rejects the inc<strong>on</strong>sistent <strong>on</strong>e; sec<strong>on</strong>d, it qualifies the data<br />

integrated into valid, doubtful, invalid in HIS and invalid in SIGLIC.<br />

In level 3, data is processed from the operati<strong>on</strong>al repository for the m<strong>on</strong>thly extracti<strong>on</strong>s, by collecting<br />

and filing data for analysis and report. At this level, the system identifies invalid data details;<br />

calculates indicators excluding the invalid data details; points out indicators with deviati<strong>on</strong>s to<br />

standards; generates warnings; data is stored in a warehouse database with the data details and<br />

business indicators that will provide the business intelligence needed to perform the governance of<br />

the system.<br />

At level 4, the informati<strong>on</strong> stored in the repositories of level 3 is processed twice a year with the first<br />

semester and annual data extracti<strong>on</strong>s. There are two steps c<strong>on</strong>cerning this level. First, after<br />

processing the first data extracti<strong>on</strong> (corresp<strong>on</strong>ding to the level 3), UCGIC team prepares a manual<br />

analysis of the data which is reported to hospitals. Then, hospitals must evaluate the informati<strong>on</strong> and<br />

correct it in their local informati<strong>on</strong> system in case there are any errors. The sec<strong>on</strong>d step is to make a<br />

sec<strong>on</strong>d data extracti<strong>on</strong> about the same period of analysis three weeks after the first extracti<strong>on</strong>. The<br />

hospitals must examine the new indicators and attach comments, while the analyst team of UCGIC<br />

evaluates the updated business intelligence together with the hospitals observati<strong>on</strong>s and produces<br />

the final reports for official publicati<strong>on</strong>.<br />

The following picture (Figure 1) represents the process of data integrati<strong>on</strong>, where the informati<strong>on</strong> is<br />

uploaded from the hospitals operati<strong>on</strong>al systems to SIGLIC operati<strong>on</strong>al database.<br />

After the data is integrated in SIGLIC operati<strong>on</strong>al database, it is validated in the 2 initial levels of the<br />

qualificati<strong>on</strong> process and stored after being certified according to SIGIC standards. By this way the<br />

model detects and limits at its beginning the integrati<strong>on</strong> of inc<strong>on</strong>sistent and n<strong>on</strong>standard data records<br />

in SIGLIC.<br />

The validati<strong>on</strong> process includes a m<strong>on</strong>itoring and management system of incidents and problems.<br />

This system allows detecting, classifying and assigning the incident/ problem to the resp<strong>on</strong>sible team<br />

in a semiautomatic way and managing its soluti<strong>on</strong> within the time defined for each type of incident/<br />

problem. The m<strong>on</strong>itoring of incidents/ problems is a totally automatic process in SIGLIC. There is also<br />

a system of communicati<strong>on</strong>s between teams including escalati<strong>on</strong> of problems, which reports the<br />

errors/incidents occurred, the communicati<strong>on</strong> to the team of the stakeholder resp<strong>on</strong>sible for its<br />

correcti<strong>on</strong> and the escalati<strong>on</strong> of the problem resoluti<strong>on</strong> if required.<br />

365


Rita Cristóvão and Pedro Gomes<br />

Figure 1: Data integrati<strong>on</strong> process<br />

SIGLIC has a standal<strong>on</strong>e versi<strong>on</strong> that can be used by hospitals as a local informati<strong>on</strong> system to<br />

manage patient care. As this applicati<strong>on</strong> is certified according to SIGIC standards, it limits the<br />

introducti<strong>on</strong> of data errors in the system by 100% and all data is integrated and qualified directly.<br />

The extracti<strong>on</strong> of data from the operati<strong>on</strong>al database to the central database in the data warehouse<br />

(DW) of SIGLIC occurs m<strong>on</strong>thly and it corresp<strong>on</strong>ds to the accumulated data since the beginning of<br />

the year. The m<strong>on</strong>thly extracti<strong>on</strong> process is represented in the next figure:<br />

Figure 2: M<strong>on</strong>thly data extracti<strong>on</strong> process<br />

366


Rita Cristóvão and Pedro Gomes<br />

The data qualificati<strong>on</strong> is made either at the level of detail and at already calculated indicators in which<br />

are excluded data details classified as invalid.<br />

In the data detail qualificati<strong>on</strong> process, the records are classified as valid according to standard<br />

values for the purpose of being included or not for calculati<strong>on</strong> of indicators. The data detail is<br />

classified as "valid", "doubtful" or “invalid” by de central system, and then return to the hospital, where<br />

the team of SIGIC maintenance, in the hospital, is resp<strong>on</strong>sible for resolving the last qualifying round.<br />

There were established procedures for qualifying the data detail for each table and field to be<br />

extracted. For each table there’s a descripti<strong>on</strong> of each field with its validati<strong>on</strong>s and qualificati<strong>on</strong>s. We<br />

can always add more tables and fields and determine its validati<strong>on</strong> according to the needs that may<br />

arise in the future.<br />

The detail elements identified as invalid in the hospital informati<strong>on</strong> system (HIS) are sent back to the<br />

hospitals for analysis and eventual correcti<strong>on</strong>. If due to this analysis the hospital c<strong>on</strong>firmed the<br />

accuracy of some of the data reported as invalid, these elements are reclassified as "valid enforced",<br />

being excluded from future automatic validati<strong>on</strong>s and reincorporated in the calculati<strong>on</strong>s of new<br />

indicators related to new extracti<strong>on</strong>s. The detail elements identified as invalid in SIGLIC are the<br />

resp<strong>on</strong>sibility of the SIGLIC maintenance team that should solve and correct them.<br />

In the indicators qualificati<strong>on</strong> process, the calculated indicators according to the valid data detail are<br />

checked with homologous variati<strong>on</strong>s and with deviati<strong>on</strong>s to the standard values by hospital service<br />

unit. For the purpose of calculating indicators, the detail elements qualified as invalid are removed<br />

from the equati<strong>on</strong>. The indicators are then calculated at the level of the hospital service unit. More<br />

validati<strong>on</strong>s may be added if necessary. At this level, indicators can be classified as "valid", "doubtful"<br />

or "highly doubtful" or even as "not qualified", when the number of elements is too small. The<br />

indicators classified as doubtful or highly doubtful are sent back to hospitals for enquiry. If due to this<br />

analysis the hospital c<strong>on</strong>firms the accuracy of some of the indicators reported as suspicious, these<br />

are reclassified as "valid enforced", being excluded from future automatic validati<strong>on</strong>s and thus<br />

presented as c<strong>on</strong>firmed in new publicati<strong>on</strong>s. This qualificati<strong>on</strong> process does not change the m<strong>on</strong>thly<br />

extracti<strong>on</strong> indicators and reports but allows the correcti<strong>on</strong> of semi-annual and annual indicators and<br />

reports, as well as for future extracti<strong>on</strong>s.<br />

Some important and more demanding reports require at least two different data extracti<strong>on</strong>s, such as<br />

the first semester and annual extracti<strong>on</strong>s. The hospital units’ are able to correct the data after the first<br />

extracti<strong>on</strong> and the ultimate analysis is c<strong>on</strong>ducted by UCGIC. The following pictures show the<br />

extracti<strong>on</strong> process of first semester and annual data.<br />

Figure 3: First semester/annual data extracti<strong>on</strong> process<br />

367


Rita Cristóvão and Pedro Gomes<br />

The first extracti<strong>on</strong> is made from the accumulated data of m<strong>on</strong>thly extracti<strong>on</strong>s that are submitted to<br />

the first and sec<strong>on</strong>d level of qualificati<strong>on</strong>. After reporting any errors of doubtful data to hospitals, the<br />

UCGIC m<strong>on</strong>itors the instituti<strong>on</strong>s feedback and any data correcti<strong>on</strong>. After a period of time defined for<br />

hospitals to perform correcti<strong>on</strong>s and c<strong>on</strong>firm the accuracy of their data, UCGIC runs a sec<strong>on</strong>d<br />

extracti<strong>on</strong> for indicators calculati<strong>on</strong> purpose, as shown in the figure below.<br />

Figure 4: Sec<strong>on</strong>d semester/annual data extracti<strong>on</strong> process<br />

At this stage the data is submitted to level 4 of the qualificati<strong>on</strong> process. Hospital units can still attach<br />

notes and explanati<strong>on</strong>s to indicators c<strong>on</strong>sidered abnormal by SIGLIC and by the UCGIC analyst<br />

team. The analyst team will c<strong>on</strong>sider the hospital observati<strong>on</strong>s and will comment <strong>on</strong> results and report<br />

them officially to the Ministry of Health. The next figure shows the diagram of data extracti<strong>on</strong> and<br />

qualificati<strong>on</strong> to provide informati<strong>on</strong> for the business intelligence (BI) system of UCGIC.<br />

Figure 5: Process of data extracti<strong>on</strong> and qualificati<strong>on</strong> for indicators/business intelligence<br />

368


Rita Cristóvão and Pedro Gomes<br />

In the indicators database there are tables with indicators daily calculated, but also weekly and<br />

m<strong>on</strong>thly. The indicators already calculated are referred to waiting list for surgery, waiting times, new<br />

entries, clinical events, surgery producti<strong>on</strong> and productivity, cancelled episodes, process c<strong>on</strong>formity,<br />

am<strong>on</strong>g others. The indicators are calculated using m<strong>on</strong>thly samples but with values grouped by day<br />

or week. These tables are updated m<strong>on</strong>thly with the extracti<strong>on</strong>s accumulated since the beginning of<br />

the year. However after being updated with the semi-annual data, the update of the informati<strong>on</strong> will be<br />

related to the sec<strong>on</strong>d semester until the annual extracti<strong>on</strong> is available. At the end of the year, all daily,<br />

weekly or m<strong>on</strong>thly indicators are updated with the annual extracti<strong>on</strong> data.<br />

The following figure represents the chr<strong>on</strong>ology of the extracti<strong>on</strong> and qualificati<strong>on</strong> process to build<br />

reports for official publicati<strong>on</strong>.<br />

Figure 6: Flow and timeline of the process of extracti<strong>on</strong> and validati<strong>on</strong> of official indicators of first<br />

semester and annual<br />

The producti<strong>on</strong> of indicators should <strong>on</strong>ly be started if the validati<strong>on</strong>s have been performed<br />

successfully. The indicators calculated will provide the hospital performance report, the operati<strong>on</strong>al<br />

dashboard of SIGLIC and the BI system QlikView.<br />

3.2 The implementati<strong>on</strong><br />

The management of the quality informati<strong>on</strong> system is carried out through SIGLIC own screens/forms<br />

and reports developed by the SIGLIC and UCGIC team. It includes a dashboard to m<strong>on</strong>itor all the<br />

extracti<strong>on</strong> and qualificati<strong>on</strong> process. Only allowed users from the teams involved in the process can<br />

access these forms in SIGLIC and restricted to the hospital unit informati<strong>on</strong> according to the users<br />

profile. The screens to manage the all process, which includes the data extracti<strong>on</strong> and qualificati<strong>on</strong><br />

process and producti<strong>on</strong> of business intelligence, are accessed through the following panel (Figure 7)<br />

in SIGLIC: From this panel, the teams involved in the extracti<strong>on</strong> and qualificati<strong>on</strong> of data can perform<br />

the following procedures/tasks:<br />

Data extracti<strong>on</strong>:<br />

Extracti<strong>on</strong>s planning: it allows planning data extracti<strong>on</strong>s. The planning states that the user can<br />

indicate the date <strong>on</strong> which extracti<strong>on</strong> should take place, for which period of time, the universe of<br />

the data to be extracted and other settings, such as data qualificati<strong>on</strong> (detail or calculated<br />

indicators), producti<strong>on</strong> of indicators for the operati<strong>on</strong>al database of SIGLIC, am<strong>on</strong>g others;<br />

Queries: it allows queries of the extracti<strong>on</strong>s performed and to be made in the future, being able to<br />

access the planning detail of each of the extracti<strong>on</strong>s.<br />

Data qualificati<strong>on</strong>:<br />

Qualificati<strong>on</strong> of data detail: it allows to access the classificati<strong>on</strong> given to each detail element, in<br />

accordance with the rules set in the qualificati<strong>on</strong> process;<br />

Qualificati<strong>on</strong> of indicators: it allows accessing the qualificati<strong>on</strong> given to each of the indicators<br />

calculated, in accordance with the requirements settled;<br />

Data detail and dashboard of indicators:<br />

Data detail query: it allows accessing to the detail of data extracted;<br />

369


Rita Cristóvão and Pedro Gomes<br />

Figure 7: Panel to access the screens of the informati<strong>on</strong> quality system in SIGLIC applicati<strong>on</strong><br />

Indicators query: dashboard of indicators available at SIGLIC;<br />

M<strong>on</strong>itoring the process of indicators producti<strong>on</strong>: it presents, by extracti<strong>on</strong>, the current status of all<br />

activities performed, as well as the list of network communicati<strong>on</strong>s and the list of incidents/<br />

problems generated in the process;<br />

Incident/ problem management: it allows managing the list of incidents/ problems in the<br />

framework of the qualificati<strong>on</strong> process. Incidents/ problems can be recorded automatically by the<br />

system or manually by users through the communicati<strong>on</strong>s network in SIGLIC. Only allowed users<br />

can register incidents/ problems. If the incidents are not resolved in the time, whose period is<br />

defined to each type of situati<strong>on</strong> by rule, shall be carried over to problems. The life cycle of<br />

incidents/ problems is also defined by rule, as well as the state diagram and the objects <strong>on</strong> which<br />

it can fall in incidents/ problems;<br />

Escalati<strong>on</strong> of problems: it presents the different levels of escalati<strong>on</strong> so far registered for the<br />

unsolved problems. The escalati<strong>on</strong> of unsolved problems is an automatic process that runs daily<br />

and is carried out according to the standard of the waiting time to solve the incident, with two<br />

goals:<br />

Transfer incidents to problems; and<br />

Escalate problems to the various levels in an organizati<strong>on</strong>.<br />

Reporting of the process workflow: Reporting about the workflow of data extracti<strong>on</strong> and<br />

qualificati<strong>on</strong> process, incident/ problem management and m<strong>on</strong>itoring is available in SIGLIC. This<br />

reporting serves as an instrument to support process c<strong>on</strong>trol, whether it is taking place within the<br />

defined times and according to the rules. This reporting includes the following informati<strong>on</strong>:<br />

Reporting of the extracti<strong>on</strong> process: performance indicators related to the extracti<strong>on</strong> process.<br />

Reporting of the qualificati<strong>on</strong> process: number of detail elements classified as invalid or doubtful;<br />

Reporting of the incident/ problem management process: number of incidents/ problems unsolved<br />

and the number of incidents/ problems dependent <strong>on</strong> a particular pers<strong>on</strong>;<br />

Reporting of the m<strong>on</strong>itoring process: informati<strong>on</strong> <strong>on</strong> the m<strong>on</strong>itoring procedures, process state,<br />

unsolved problems, etc.<br />

370


4. Results<br />

Rita Cristóvão and Pedro Gomes<br />

The model for informati<strong>on</strong> quality in elective surgery is being implemented since the beginning of<br />

2011 and will be finished by the end of 2012. Most of the manual procedures that exist nowadays are<br />

going to be automated procedures performed by SIGLIC. The automated tasks for data extracti<strong>on</strong> and<br />

qualificati<strong>on</strong> at level 1 and 2 are already implemented and we can see important impacts and<br />

outcomes of this new approach to guarantee reliable informati<strong>on</strong> in SIGLIC with lower costs in human<br />

resources. By this new model, UCGIC is already assuring that the hospital performance indicators of<br />

the existing dashboard in SIGLIC operati<strong>on</strong>al database are correct and reflect the actual care<br />

provided to patients in real time, the hospital performance according to care provided and allows the<br />

accurate evaluati<strong>on</strong> of demand and supply in elective surgery. Additi<strong>on</strong>ally, the dashboard is<br />

uploaded with informati<strong>on</strong> from the hospital operati<strong>on</strong>al databases in real time, so the indicators are<br />

not <strong>on</strong>ly reliable but actual. Also this automatic process is c<strong>on</strong>siderably less time and resource<br />

c<strong>on</strong>suming. Studies c<strong>on</strong>ducted by UCGIC about the implemented soluti<strong>on</strong>s c<strong>on</strong>cluded that we are<br />

saving nearly 5 days to a process which took 10 days. In fact, by allowing automatic reports of errors<br />

to the teams involved in the process, the incident and problem management is a much more efficient<br />

process which turns its resoluti<strong>on</strong> more easily and quickly with the hospital units.<br />

Another result of the partial implementati<strong>on</strong> of the informati<strong>on</strong> quality model is the ability to provide<br />

customized reports <strong>on</strong> demand to several entities. With the development of the new features and<br />

screens in SIGLIC, in 2011 UCGIC was able to satisfy an increasing number of indicators and reports<br />

requests and c<strong>on</strong>duct more regular reports to m<strong>on</strong>itor hospitals performance. The volume of work in<br />

providing indicators <strong>on</strong> demand increased 64% compared with 2010. This is an area that will c<strong>on</strong>tinue<br />

to grow and where the UCGIC is investing greatly in the qualificati<strong>on</strong> of human resources and<br />

informati<strong>on</strong> technology.<br />

The next table shows the evoluti<strong>on</strong> of performance indicators of SIGLIC related to the quality<br />

informati<strong>on</strong> system from 2010 to 2011.<br />

Table 2: Performance indicators of SIGLIC related to the informati<strong>on</strong> quality system<br />

2010 2011<br />

Annual<br />

Variati<strong>on</strong><br />

SIGLIC maintenance Nr incidents/ problems reported to the<br />

helpdesk<br />

4.888 18.321 275%<br />

Nr of working hours in the applicati<strong>on</strong><br />

(Nr of entrees times access time)<br />

520.896 789.222 52%<br />

Nr of users 6.611 7.495 13%<br />

Nr of data transacti<strong>on</strong>s 4.853.236 5.857.978 21%<br />

Nr of collaborators in UCGIC 11 11 0%<br />

Nr of collaborators in hospitals 32.365 38.202 18%<br />

Nr of hospital units in SIGIC 116 113 -3%<br />

DW Total Volume 950GB 1441GB 52%<br />

Annual growth of DW Total Volume<br />

Nr of clinical processes managed by<br />

450GB 490GB -<br />

SIGLIC (entries in waiting list for<br />

surgery)<br />

565.971 600.331 6%<br />

Nr input variables 881 881 0%<br />

Indicators/Reports Nr of reports c<strong>on</strong>ducted by UCGIC,<br />

planned and <strong>on</strong>-demand<br />

45 74 64%<br />

5. C<strong>on</strong>clusi<strong>on</strong>s<br />

The model to guarantee informati<strong>on</strong> quality developed in SIGLIC by UCGIC provides an innovative<br />

process for validati<strong>on</strong> and qualificati<strong>on</strong> of massive data extracted from various local informati<strong>on</strong><br />

systems. It includes automated processes of detecti<strong>on</strong> and reporting of errors, correcti<strong>on</strong> of errors in<br />

source data and c<strong>on</strong>tinuous m<strong>on</strong>itoring of the process mostly automated according to best practices<br />

in informati<strong>on</strong> quality. By turning the informati<strong>on</strong> quality system managed automatically by SIGLIC, it<br />

is much less time and resource c<strong>on</strong>suming, allowing UCGIC to provide reliable informati<strong>on</strong> to all<br />

entities through customized reports <strong>on</strong> demand, the dashboard updated by SIGLIC operati<strong>on</strong>al<br />

database and the UCGIC BI system powered by the official data for publicati<strong>on</strong>. In 2011 UCGIC was<br />

able to provide indicators and reports <strong>on</strong> demand by more than 64% compared with 2010. Informati<strong>on</strong><br />

quality is essential to produce reports about historical, current and predictive analysis of elective<br />

371


Rita Cristóvão and Pedro Gomes<br />

surgery producti<strong>on</strong>, which will c<strong>on</strong>duct to the creati<strong>on</strong> of knowledge and intelligence to support better<br />

decisi<strong>on</strong>-making by all stakeholders involved, including the government with its policies.<br />

The challenge is to c<strong>on</strong>tinue to improve the informati<strong>on</strong> quality system according to the real needs of<br />

all stakeholders involved, from the source of the informati<strong>on</strong> in hospitals to the end-user of the<br />

indicators and reports. For that, UCGIC has to c<strong>on</strong>stantly adapt its structure and resources to satisfy<br />

its client’s informati<strong>on</strong> needs. Another challenge is to create an efficient system to certify informati<strong>on</strong><br />

quality in SIGIC, especially in local hospital systems, and improve c<strong>on</strong>tinuously the defined standards<br />

according to new realities. The capacity of adaptati<strong>on</strong> is the key factor for the success of this<br />

innovative informati<strong>on</strong> quality system.<br />

372


PhD<br />

Research<br />

Papers<br />

373


374


Informati<strong>on</strong> Risks and Their C<strong>on</strong>necti<strong>on</strong> With Accounting<br />

Marie Černá<br />

Faculty of Ec<strong>on</strong>omics - University of West Bohemia, Pilsen, Czech Republic<br />

macerna@kfu.zcu.cz<br />

Abstract: Risks can occur at any stage of the producti<strong>on</strong> process or the provisi<strong>on</strong> of mediati<strong>on</strong> services.<br />

Informati<strong>on</strong> are then important basis for all decisi<strong>on</strong>-making processes that take place in companies. They<br />

represent <strong>on</strong>e of the most scarce and important assets. Risks can be found at every step and the ability to<br />

recognize, describe and analyze them is (especially for managers and executives) very important. In order to be<br />

able to manage risks effectively, we use the help of risk analysis. It is divided into several parts, of which the first<br />

<strong>on</strong>e, risk identificati<strong>on</strong>, represents a very difficult matter. Proper identificati<strong>on</strong> of risk is an essential comp<strong>on</strong>ent for<br />

the correct implementati<strong>on</strong> of other parts of the risk analysis and the next steps in the process of risk<br />

management, and finally also for finding an appropriate method of risk management in c<strong>on</strong>diti<strong>on</strong>s of a particular<br />

company. In practice, it is possible to meet many types of risks. They are usually not isolated. We can meet them<br />

as a complex of several types of risks. Am<strong>on</strong>g the risks that are currently often discussed (in c<strong>on</strong>necti<strong>on</strong> with<br />

increasing importance of informati<strong>on</strong> and development of informati<strong>on</strong> technologies), are included informati<strong>on</strong><br />

risks. This type of risk has been studied in relati<strong>on</strong> to many areas of human activity. Accounting is a scientific<br />

discipline, whose faultless functi<strong>on</strong>ing in practice is significant for ensuring smooth running of the company.<br />

Better utilizati<strong>on</strong> and adaptati<strong>on</strong> of existing informati<strong>on</strong> coming from managerial and financial accounting to the<br />

needs of risk management can be seen as a topic of future potential. Therefore, I believe that the analysis of<br />

including informati<strong>on</strong> risk management am<strong>on</strong>g other business activities being c<strong>on</strong>ducted within this area, is a<br />

topic which would be appropriate to c<strong>on</strong>tinue to deal with in detail. This study describes the relati<strong>on</strong>ship between<br />

accounting and informati<strong>on</strong> management. The analysis is prepared more theoretically using deducti<strong>on</strong> and<br />

literature review. The findings of proposed qualitative research may be used as a starting point for more<br />

extensive research.<br />

Keywords: informati<strong>on</strong>, risks, informati<strong>on</strong> risks, risk analysis, risk management, accounting<br />

1. Introducti<strong>on</strong><br />

Informati<strong>on</strong>, risks, risk management and other terms c<strong>on</strong>nected with business process management<br />

represent often discussed topics. One of the reas<strong>on</strong>s for such situati<strong>on</strong> is the fact that risks affect all<br />

daily activities c<strong>on</strong>ducted in c<strong>on</strong>necti<strong>on</strong> with ensuring business processes. Risks are usually not<br />

isolated. In most cases, we can meet them as a complex of many different types of risks, which have<br />

to be managed carefully. Am<strong>on</strong>g the risks that are nowadays often menti<strong>on</strong>ed, can be included the<br />

informati<strong>on</strong> risk. <strong>Management</strong> of informati<strong>on</strong> risks represents the subject of study of relatively recently<br />

(ten to twenty years ago) established scientific discipline called informati<strong>on</strong> management that is<br />

currently seen as a necessary part of the company risk management.<br />

Informati<strong>on</strong> risks are often described <strong>on</strong>ly as the risks c<strong>on</strong>nected with the security of informati<strong>on</strong><br />

systems. They represent wider issue with number of different perspectives. It is possible to meet here<br />

c<strong>on</strong>cepts and knowledge known from other scientific disciplines, like for example human resource<br />

management, project management, risk management, software engineering, team work<br />

management, change management, etc. Informati<strong>on</strong> risks, the same way as other types of risks, can<br />

be met in all areas of human activity. We use the help of risk management to manage informati<strong>on</strong><br />

risks effectively. The first part of risk management is risk analysis. It is divided into several parts, of<br />

which the first <strong>on</strong>e, risk identificati<strong>on</strong>, represents a very difficult matter. Proper identificati<strong>on</strong> of risk is<br />

an essential comp<strong>on</strong>ent for the correct implementati<strong>on</strong> of other parts of the risk analysis and the next<br />

steps in the process of risk management, and finally also for finding an appropriate method of risk<br />

management in c<strong>on</strong>diti<strong>on</strong>s of a particular company. Existence of informati<strong>on</strong> systems enabled us to<br />

deal with risk management using more effective way, because they help us to process a large amount<br />

of data in a relatively short time and they provide a huge amount of informati<strong>on</strong>, which are necessary<br />

for managing risks. Through it or just because of that, we can still meet new problems and topics<br />

c<strong>on</strong>nected with risk management. It seems to be necessary to cooperate with experts from other<br />

fields, not <strong>on</strong>ly informatics, but others, like for example accountants, to solve these topics.<br />

2. Background<br />

2.1 Informati<strong>on</strong><br />

Informati<strong>on</strong> can be explained using two different points of view. The first of them emphasizes the fact<br />

that informati<strong>on</strong> is in principle seen as objectively existing, c<strong>on</strong>stant and independent value (entity).<br />

375


Marie Černá<br />

Such a c<strong>on</strong>cept is the same as the definiti<strong>on</strong> of sources like m<strong>on</strong>ey or material. The questi<strong>on</strong>, how to<br />

evaluate this source, is sec<strong>on</strong>dary. Primary problem is the availability of this type of source. If we use<br />

the first type of explanati<strong>on</strong>, we basically accept the idea that to use generally applicable and solvable<br />

universal informati<strong>on</strong> systems is possible. Informati<strong>on</strong> management can be then limited to work with<br />

data sources. Informati<strong>on</strong> are most often seen this way within the hard (engineering) approaches. The<br />

sec<strong>on</strong>d point of view emphasizes that the classical sources are <strong>on</strong>ly the holders of informati<strong>on</strong>, data.<br />

Informati<strong>on</strong> is seen as the purposeful selecti<strong>on</strong> of data that is dependent <strong>on</strong> the dispositi<strong>on</strong> of the<br />

user, his subjective needs and his ability to interpret the informati<strong>on</strong>. This sec<strong>on</strong>d point of view does<br />

not expect direct transfer of informati<strong>on</strong> using informati<strong>on</strong> systems. It is menti<strong>on</strong>ed as the provisi<strong>on</strong> of<br />

data that represent the support of the managerial work (Vodáček-Rosický 1997). The meaning of<br />

informati<strong>on</strong> is important for this approach and this is typical for social systems. In practice<br />

organizati<strong>on</strong>s usually use both starting points, but in varying degrees.<br />

Table 1: The influence of the c<strong>on</strong>cept of informati<strong>on</strong> management approaches (Vodáček-Rosický<br />

1997)<br />

The influence of the c<strong>on</strong>cept of informati<strong>on</strong> management approaches<br />

The mechanistic approach of<br />

organizati<strong>on</strong> views the informati<strong>on</strong> as<br />

the objective source.<br />

The organistic approach emphasizes<br />

informati<strong>on</strong> as the interpretati<strong>on</strong> of<br />

data.<br />

Focused <strong>on</strong> Producti<strong>on</strong>, quantity and efficiency Adaptability, quality and usefulness<br />

Organizati<strong>on</strong>al complexity Low High<br />

Centralizati<strong>on</strong> High Low<br />

Formalizati<strong>on</strong> High Low<br />

Number of levels Higher and clear Lower, often unclear<br />

Adaptivity Low and short-term High and l<strong>on</strong>g-term<br />

Output High and short-term Low with growth perspective<br />

Work is significantly differentiated<br />

and specialized. Publicity of the<br />

relati<strong>on</strong>ship between the work tasks<br />

and the aims of the organizati<strong>on</strong> is<br />

not desirable.<br />

Work tasks are strictly defined<br />

(rights, methods and technological<br />

procedures are specified in detail for<br />

each staff or functi<strong>on</strong>al positi<strong>on</strong>).<br />

They are fixed and can be changed<br />

<strong>on</strong>ly by the decisi<strong>on</strong> of the highest<br />

level of c<strong>on</strong>trol.<br />

Regulatory resp<strong>on</strong>sibility has a<br />

strictly hierarchical structure.<br />

Working behavior is derived from the<br />

c<strong>on</strong>tract between the organizati<strong>on</strong><br />

and an employee.<br />

Informati<strong>on</strong> important for<br />

management are accepted by the<br />

organizati<strong>on</strong> and transferred to the<br />

chief executive officer.<br />

Communicati<strong>on</strong> is mostly vertical.<br />

Crucial c<strong>on</strong>tent of communicati<strong>on</strong> is<br />

two fold:<br />

- commands, decisi<strong>on</strong>s and the<br />

superiors instructi<strong>on</strong>s,<br />

- reports and supporting informati<strong>on</strong><br />

for decisi<strong>on</strong> making (made by<br />

subordinates).<br />

Work tasks are interc<strong>on</strong>nected. Clear<br />

interc<strong>on</strong>necti<strong>on</strong> of the major tasks<br />

and business strategy is<br />

emphasized.<br />

General functi<strong>on</strong>s are defined:<br />

employees use the individual<br />

approach to fulfilling their obligati<strong>on</strong>s<br />

according with given standards. They<br />

take their pers<strong>on</strong>al resp<strong>on</strong>sibilities<br />

and encourage redefiniti<strong>on</strong> of given<br />

tasks, relati<strong>on</strong>ships and needs.<br />

Regulatory resp<strong>on</strong>sibility is<br />

decentralized and network structure<br />

of relati<strong>on</strong>s exits. Working behavior<br />

takes into account the interests of<br />

the organizati<strong>on</strong> and the individual<br />

that accepts participati<strong>on</strong>.<br />

Important informati<strong>on</strong> are shared and<br />

teamwork that increases the base of<br />

knowledge for decisi<strong>on</strong> making is<br />

often applied.<br />

Communicati<strong>on</strong> is vertical and<br />

horiz<strong>on</strong>tal dependent <strong>on</strong> informati<strong>on</strong><br />

needs and reachable data sources. It<br />

takes into account competencies and<br />

knowledge of individuals.<br />

Communicati<strong>on</strong> has a form of<br />

sharing informati<strong>on</strong>, transfer of<br />

knowledge and advice. It includes<br />

also sharing ideas, values and<br />

experiences.<br />

Informati<strong>on</strong> are nowadays viewed by the companies as a specific type of asset. They represent<br />

valuable asset that has to be treated carefully, because they are used as a base for each decisi<strong>on</strong><br />

making process that takes place inside the company and that helps companies to achieve their given<br />

goals. What is interesting about the informati<strong>on</strong> is the fact that they d<strong>on</strong>´t lose their value due to effect<br />

376


Marie Černá<br />

of time. On the c<strong>on</strong>trary, their value could be increased due to such an effect. To be able to indicate<br />

informati<strong>on</strong> as useful for final users, we have to be sure that they meet the qualitative characteristics<br />

which are their relevance, reliability, comparability and understandability.<br />

As the bearer of informati<strong>on</strong>, we can identify the numeric data, text, sound, image, … Informati<strong>on</strong><br />

represent the base for knowledge creati<strong>on</strong>. Knowledge is sometimes described as the informati<strong>on</strong><br />

that are represented and transferred as data or signals. Further progress is the percepti<strong>on</strong> of<br />

informati<strong>on</strong> not <strong>on</strong>ly as the signals, but also as the symbols. This point of view indicates<br />

understanding of the informati<strong>on</strong> system as a cultural phenomen<strong>on</strong> rather than a technical<br />

achievement.<br />

Meaning of the terms informati<strong>on</strong> and knowledge overlaps in this article. Within the researches that<br />

are currently made and articles about informati<strong>on</strong> management that are written, is such a c<strong>on</strong>cept<br />

often found. Then we focus <strong>on</strong> the fact that informati<strong>on</strong> can be understood as the valuable asset, but<br />

also as the source of risk. Problematic situati<strong>on</strong> occurs, when we have lack of informati<strong>on</strong> in decisi<strong>on</strong><br />

making, but the same risks may arise, when we have too much informati<strong>on</strong>.<br />

2.2 Risk<br />

Risk is the term that can be already met in history. The use of this term in c<strong>on</strong>necti<strong>on</strong> with shipping is<br />

known since the seventeenth century. It comes from the Italian „risico“. First we meet it in<br />

mathematics and later in ec<strong>on</strong>omic subjects. In ec<strong>on</strong>omic subjects is risk seen mainly as a possible<br />

loss. Nowadays it is understood mainly as a possibility that the loss occurs or as a possibility of failure<br />

during c<strong>on</strong>ducted activities. We have no generally accepted definiti<strong>on</strong> for the term „risk“. That is why<br />

we can find a lot of definiti<strong>on</strong>s that tries to specify the risk. Here are some of them:<br />

„Risk is the probability of any result different from the expected <strong>on</strong>e.“ (Smejkal-Rais<br />

2006)<br />

„Risk is an uncertain event or c<strong>on</strong>diti<strong>on</strong>, whose incidence has a positive or a negative<br />

impact <strong>on</strong> project objectives.“ (Svozilová 2007)<br />

Risk represents the change of profit caused by deviati<strong>on</strong> of the m<strong>on</strong>itored parameter from the planned<br />

status. It is usually expressed in terms of impact, expressed in m<strong>on</strong>etary units, and percentage<br />

probability of occurence in a given time period. An uncertain outcome and the possibility of achieving<br />

at least <strong>on</strong>e undesirable result is c<strong>on</strong>nected with risk. The risk is usually understood as a negative<br />

impact <strong>on</strong> given activities. On the other hand, it is also possible to see the risk as a positive thing, as<br />

an incentive or motivati<strong>on</strong> to find better alternative soluti<strong>on</strong> of any problem. The risk therefore has two<br />

aspects, positive and negative. As was written in the introducti<strong>on</strong>, we can identify many types of risk<br />

and many possibilities how to classify them.<br />

Table 2: Possible classificati<strong>on</strong> of risks<br />

Possible classificati<strong>on</strong> of risks<br />

Internal and external<br />

Subjective and objective<br />

Frequently occurring and rare<br />

Financial and n<strong>on</strong>-financial<br />

Risk<br />

Static and speculative<br />

Stable and instable<br />

Financial risks, security risks and envir<strong>on</strong>mental protecti<strong>on</strong> risks, risks of informati<strong>on</strong><br />

systems, project risks, …<br />

…<br />

Besides the methods of classificati<strong>on</strong> menti<strong>on</strong>ed above, we can find many others that differ author by<br />

author.<br />

For example informati<strong>on</strong> risks (risks of informati<strong>on</strong> systems) can be further divided to:<br />

Risks c<strong>on</strong>nected with the possibility that the quality of informati<strong>on</strong> will be threatened.<br />

Risks arising from the loss of informati<strong>on</strong>.<br />

Failure during the work with informati<strong>on</strong>.<br />

377


Marie Černá<br />

Each risk has its source, its cause and is c<strong>on</strong>nected with specific symptoms and effects. Sources and<br />

causes of informati<strong>on</strong> risks can be unclearly defined project objectives or project objectives defined by<br />

another pers<strong>on</strong> than the final user, different or unrealistic expectati<strong>on</strong>s of managers and product users<br />

or ambiguous requirements for initial informati<strong>on</strong>. The final effects are then deviati<strong>on</strong>s from the setted<br />

project objectives, time delays, different utilizati<strong>on</strong> rate of final informati<strong>on</strong> system possibilities by the<br />

user or problems with utilizati<strong>on</strong> of all possibilities that gives such informati<strong>on</strong> system to the user and<br />

return to the previous informati<strong>on</strong> system. To be able to work with risks and to manage them, we use<br />

the risk management tools.<br />

2.3 Risk management<br />

The competiti<strong>on</strong> between companies is c<strong>on</strong>stantly growing and evolving. It becomes more complex.<br />

Nowadays we can meet the theories that work with the term „supercompetiti<strong>on</strong>“ that changes recently<br />

established relati<strong>on</strong>ships.<br />

Table 3: Relati<strong>on</strong>s before and after the supercompetiti<strong>on</strong> (Jirásek 2008)<br />

Relati<strong>on</strong>s before the supercompetiti<strong>on</strong> Relati<strong>on</strong>s after the supercompetiti<strong>on</strong><br />

Demand ec<strong>on</strong>omy Ec<strong>on</strong>omy of supply<br />

Market equilibrium Market imbalance<br />

Sustainable advantage Unsustainable advantage<br />

Market research New market creati<strong>on</strong><br />

Profit Value added, cash flow<br />

Standard producti<strong>on</strong> Order producti<strong>on</strong><br />

Quid pro quo The winner takes all<br />

People Pers<strong>on</strong>alities<br />

Tasks Authorizati<strong>on</strong> and motivati<strong>on</strong><br />

Employment Employability<br />

Informati<strong>on</strong> Knowledge<br />

Hierarchical organizati<strong>on</strong>al structure Informal organizati<strong>on</strong>al structure<br />

Designing Modeling<br />

Companies meet risks very often in such an envir<strong>on</strong>ment. It´s important for them to be able to set the<br />

priorities in risk management with respect to their impact and probability of their occurence and to<br />

focus <strong>on</strong> those risks that can be seen as the key risks in terms of business management. Risk<br />

management represents an important part of strategic management of each company. It is a complex<br />

process that is seen as a subject of c<strong>on</strong>tinuous development, improvement and allows companies to<br />

focus <strong>on</strong> risks c<strong>on</strong>nected with all activities provided by them. Important questi<strong>on</strong>s that should be<br />

discussed in c<strong>on</strong>necti<strong>on</strong> with risk management are:<br />

What do we want to eliminate?<br />

What do we want to c<strong>on</strong>trol?<br />

Risk management process c<strong>on</strong>sists of several steps. It c<strong>on</strong>sists of the risk analysis (identificati<strong>on</strong> of<br />

assets, determining the value of assets, identificati<strong>on</strong> of threats, weaknesses, determinati<strong>on</strong> of<br />

relevance of threats and weaknesses), risk assessment and risk management. All these comp<strong>on</strong>ents<br />

of risk management are important and their processing requires accuracy and c<strong>on</strong>tinuous m<strong>on</strong>itoring<br />

of c<strong>on</strong>ducted operati<strong>on</strong>s. Incorrect implementati<strong>on</strong> of any part can lead not <strong>on</strong>ly to poorly defined and<br />

assessed risks, but also to failure in future business plans.<br />

It is the same in the case of informati<strong>on</strong> risks. If we are not able to prepare correctly the first part,<br />

identificati<strong>on</strong> of risks, we can for example unc<strong>on</strong>sciously endanger the implementati<strong>on</strong> of business<br />

informati<strong>on</strong> system or activities that will be ensured by the final user of such system. In a case of the<br />

employee of business department can be met the situati<strong>on</strong>, when it will be necessary to find out the<br />

state of the sales of given product immediately, clearly print these data and give them in an usable<br />

form to the manager to assess them. If there are not clearly specified requirements at the beginning<br />

(before the implementati<strong>on</strong> of informati<strong>on</strong> system) or if the requirements are specified by other pers<strong>on</strong><br />

than the final user of this informati<strong>on</strong> system without taking into account comments of the final user<br />

that is expected to prepare and print final reports, we will most probably meet the situati<strong>on</strong> that the<br />

informati<strong>on</strong> system w<strong>on</strong>´t meet the expectati<strong>on</strong>s of the user. For example it will be necessary to<br />

transfer reports from <strong>on</strong>e program to program or to prepare them manually and to calculate some<br />

378


Marie Černá<br />

data and then transfer them back to informati<strong>on</strong> system and to print final reports without the possibility<br />

to ensure these activities using informati<strong>on</strong> system module. Such a situati<strong>on</strong> may lead to<br />

dissatisfacti<strong>on</strong> of the final user of the system or manager and to percepti<strong>on</strong> of the informati<strong>on</strong> system<br />

as a tool that does not facilitate the work, rather the opposite. That is why it is so important to involve<br />

in the process of informati<strong>on</strong> risks management the managers, but also other employees that will<br />

probably use this informati<strong>on</strong> system in the future.<br />

2.4 Informati<strong>on</strong> management<br />

The term „informati<strong>on</strong> management“ has no clear definiti<strong>on</strong>. It has many reas<strong>on</strong>s, but widely spoken<br />

are three of them:<br />

ambiguous understanding of the term „management“,<br />

ambiguous understanding of the term „informati<strong>on</strong>“,<br />

Effect of the change in understanding the term „informati<strong>on</strong> management“ during the period since<br />

its establishment (Vodáček-Rosický 1997).<br />

Informati<strong>on</strong> management passed through three development stages. This term was used for the first<br />

time by R. S. Taylor in the year 1966 during the c<strong>on</strong>ferenti<strong>on</strong> focused <strong>on</strong> the questi<strong>on</strong>s of system<br />

approach and processing of engineering informati<strong>on</strong> and educati<strong>on</strong>. At that time, informati<strong>on</strong><br />

management was seen as a discipline primarily focused <strong>on</strong> solving technical problems. The sec<strong>on</strong>d<br />

phase that is represented by the period between the seventieth and eightieth was focused <strong>on</strong> the<br />

efficiency of labor in the ec<strong>on</strong>omy. We can find transdisciplinary approach to informati<strong>on</strong><br />

management, interc<strong>on</strong>necti<strong>on</strong> of informatics and management views <strong>on</strong> the given issue here. Main<br />

tasks of managerial activities that are found through questi<strong>on</strong>s that discuss, which possibilities<br />

informati<strong>on</strong> technologies have to support ensuring objectives of the organizati<strong>on</strong>, still remained in the<br />

background. The emphasis was given to ensuring efficient informati<strong>on</strong> processes. Beginning of the<br />

nineties represents the third phase in informati<strong>on</strong> management development. The emphasis is now <strong>on</strong><br />

effective achievement of objectives of the organizati<strong>on</strong> using computing and informati<strong>on</strong> technologies.<br />

In this phase the need to ensure that all managerial activities will be made effectively (effectiveness –<br />

doing the right things) and efficiently (doing the things right) is also perceived. Currently we see<br />

informati<strong>on</strong> management as a discipline, where applicati<strong>on</strong>s of informati<strong>on</strong> systems and applicati<strong>on</strong>s<br />

of informati<strong>on</strong> technologies d<strong>on</strong>´t represent goals of the manager. They are used by managers as an<br />

effective tool that helps them to improve their activities by ensuring their individual informati<strong>on</strong> needs.<br />

This can be seen as the c<strong>on</strong>necti<strong>on</strong> between modern management, informatics, system approaches<br />

and views of other disciplines like ec<strong>on</strong>omics, psychology etc. Informati<strong>on</strong> management has a wide<br />

range of use (industrial sectors, educati<strong>on</strong>, health care system, …).<br />

Currently we are talking about knowledge management or requirements management rather than<br />

about informati<strong>on</strong> management within the risk management. This happens probably in c<strong>on</strong>necti<strong>on</strong><br />

with the transfer from the learning enterprise to the knowledge-based enterprise. For such<br />

management it is necessary to have the functi<strong>on</strong>al informati<strong>on</strong> system that enables us to detect,<br />

m<strong>on</strong>itor and measure possible risks.<br />

Table 4: Differences between the learning enterprise and knowledge-based enterprise (Jirásek 2008)<br />

Learning enterprise Knowledge-based enterprise<br />

Work with informati<strong>on</strong> Collects, accumulates and uses<br />

Work with knowledge<br />

Collects, accumulates and forms the<br />

new<br />

3. Informati<strong>on</strong> management and accounting<br />

As was written above, informati<strong>on</strong> management is a transdisciplinary scientific discipline. The close<br />

relati<strong>on</strong>ship of informati<strong>on</strong> management to many disciplines is known, but we also have to say that its<br />

relati<strong>on</strong>ship to some disciplines is not menti<strong>on</strong>ed very often. Accounting bel<strong>on</strong>gs to that sec<strong>on</strong>d group.<br />

Without being aware of it, accounting is a discipline, whose activities influence many other areas of<br />

human activity. It provides important informati<strong>on</strong> about the current market situati<strong>on</strong> of supplied<br />

products or services, informati<strong>on</strong> about the management within the company and many others. In the<br />

situati<strong>on</strong> when it is increasingly difficult to succeed in a competitive market envir<strong>on</strong>ment, the<br />

companies are looking for opportunities to help them at least to maintain their market positi<strong>on</strong>.<br />

379


Marie Černá<br />

Michael Porter, who studied the strategy and competitiveness, described two alternatives for<br />

achieving better competitive positi<strong>on</strong> of the company, diversity and cost reducti<strong>on</strong>. Whereas the<br />

achievement of diversity seems to be more difficult, companies tried to use the sec<strong>on</strong>d way, cost<br />

reducti<strong>on</strong>, in the past. Currently the possibilities provided by the cost reducti<strong>on</strong> aren´t the same as<br />

they were years ago. That is why other possibilities that will allow the companies not <strong>on</strong>ly to hold, but<br />

also to improve their future market positi<strong>on</strong> are c<strong>on</strong>sidered. The possibility of changing the generally<br />

accepted social values are even discussed in c<strong>on</strong>necti<strong>on</strong> with that.<br />

In the past accounting was established to be able to match the given state standards. The care was<br />

taken to ensure the accounting to be kept correctly and no complaint from the supervisor, the state,<br />

were met. It was basically current financial accounting. The need to use accounting another way, as<br />

the source of advice in ec<strong>on</strong>omics or as the help to managers of the companies, arised later. The<br />

reas<strong>on</strong> for this was in interpenetrati<strong>on</strong> of accounting activities to business. Nowadays the existence of<br />

accounting informati<strong>on</strong> system that is prepared to be used the way menti<strong>on</strong>ed above represents the<br />

standard issue (comm<strong>on</strong>ly known thing), especially in a case of large enterprises. Informati<strong>on</strong> that<br />

have to be provided to managers by accounting are more or less standardized. What can be different<br />

is the way of their acquisiti<strong>on</strong> and transfer to manager. Final reports are, in terms of managerial<br />

accounting, delivered in various forms. To be able to ensure the required format of the final report, the<br />

cooperati<strong>on</strong> of accountants (ec<strong>on</strong>omists) and informatics is necessary. For the first sight it seems to<br />

be the situati<strong>on</strong> that brings no possibility to meet any kind of risk, but it is just an illusi<strong>on</strong>. Risks can<br />

occur in any phase of the risk management process, whose most expected part is its implementati<strong>on</strong>.<br />

Underestimati<strong>on</strong> of any part of risk management process may lead to the disclosure of unpleasant<br />

facts as was written in the previous part of this article. To be able to avoid such an unpleasant<br />

situati<strong>on</strong>, we can use the help of informati<strong>on</strong> management tools.<br />

It seems that informati<strong>on</strong> management is clearly described discipline that is well-known to managers.<br />

The reality is often different. If you ask managers of small and medium sized enterprises what<br />

informati<strong>on</strong> management is, the answer is in many cases wr<strong>on</strong>g. In my thesis I will try to describe<br />

management, risk and informati<strong>on</strong>. I will also analyze and evaluate the current situati<strong>on</strong> of integrati<strong>on</strong><br />

of informati<strong>on</strong> risks to complex risk management of the selected subjects. I will try to verify possible<br />

relati<strong>on</strong>s between informati<strong>on</strong> risk management and other disciplines (accounting) and their impact <strong>on</strong><br />

improving the business activities. This should be d<strong>on</strong>e with respect not <strong>on</strong>ly to tools, but also to soft<br />

factors, for example to skills and knowledge of final users of informati<strong>on</strong> system, to their<br />

communicati<strong>on</strong> skills, their current physical and mental state, etc.<br />

My research is based <strong>on</strong> collecting, analyzing and interpreting informati<strong>on</strong> c<strong>on</strong>nected with possible<br />

cooperati<strong>on</strong> of informatics and experts from other fields (accountants). To reach the aim of my thesis<br />

(Descripti<strong>on</strong> of informati<strong>on</strong> risks and identificati<strong>on</strong> of problems c<strong>on</strong>nected with them in activities of<br />

selected business entities.), I use the methods like literature review, descripti<strong>on</strong>, analysis, synthesis,<br />

inducti<strong>on</strong>, deducti<strong>on</strong>, qualitative research and interview (semi-structured, 45 minutes). The final output<br />

should be prepared using qualitative method of research (expert interview, small number of<br />

resp<strong>on</strong>dents). Used questi<strong>on</strong>s should start with:<br />

Who?<br />

Why?<br />

How?<br />

The first thing I had to do was preparing the questi<strong>on</strong>s for expert interviews and decide the type and<br />

amount of resp<strong>on</strong>dents. Because of the fact that I expect semi-structured expert interviews, I decided<br />

to c<strong>on</strong>tact accountants and informatics from 5 – 10 organizati<strong>on</strong>s (small and medium sized<br />

enterprises) in West Bohemia. The questi<strong>on</strong>s were divided into four groups:<br />

general informati<strong>on</strong> (name of the company, status, head office, …),<br />

informati<strong>on</strong>, informati<strong>on</strong> system (organizati<strong>on</strong>al structure of the company, type of used informati<strong>on</strong><br />

system, which problems arise during the use of informati<strong>on</strong> system, …),<br />

accounting data, accounting informati<strong>on</strong> system (number of accountants, type of accounting<br />

informati<strong>on</strong> system, user rights, users of informati<strong>on</strong>, compatibility of accounting informati<strong>on</strong><br />

system and informati<strong>on</strong> system of the organizati<strong>on</strong>, …),<br />

Risks, risk management (use of informati<strong>on</strong> management, risk analysis – assets and risk<br />

factors,…).<br />

380


Marie Černá<br />

I expect that this research will help me with writing the final report about informati<strong>on</strong> system user<br />

requirements and activities as accurately as possible. Expert interviews c<strong>on</strong>ducted with boths,<br />

accountants and informatics, should help me to detect and classify the main problems c<strong>on</strong>nected with<br />

their cooperati<strong>on</strong>. I have d<strong>on</strong>e some of those interviews and I can say that there are many similarities<br />

between the studied companies. One of them is the fact that they usually use the tools of informati<strong>on</strong><br />

management instinctively and this can cause problems (no written statement - omissi<strong>on</strong> of important<br />

asset, underestimati<strong>on</strong> of any kind of risk, analysis <strong>on</strong>ly in the middle part of the project (activity), …).<br />

The final output of suggested qualitative research should bring also new ideas for further<br />

(quantitative) research. The aim of my thesis is not to do the statistical research, but to obtain credible<br />

and realistic view of studied issue. That is why I prefer the form of qualitative research. The<br />

subjectivity of resp<strong>on</strong>dent´s answers should be seen as a problem, but we can also see the positive<br />

effect, because the aim is to focus <strong>on</strong> problems c<strong>on</strong>nected with practical use of informati<strong>on</strong> and<br />

informati<strong>on</strong> systems. Those shouldn´t be menti<strong>on</strong>ed in any available literature.<br />

4. C<strong>on</strong>clusi<strong>on</strong>s and outlook<br />

The existence of computer and internet implies the possibility of more comfortable work with data,<br />

informati<strong>on</strong> and knowledge. The possibility to process huge amounts of data in a relatively short<br />

period is ensured that way. Many years ago, similar functi<strong>on</strong>s were provided by different technologies<br />

and other media were used. Historical and current informati<strong>on</strong> systems d<strong>on</strong>´t differ in the way of<br />

functi<strong>on</strong>ing, but in the used technology. Computer is now a routine part of each business. Existing and<br />

functi<strong>on</strong>ing informati<strong>on</strong> system represents the necessary prerequisite of quality setting of business<br />

processes and their c<strong>on</strong>trol. This enables the managers quick orientati<strong>on</strong> in c<strong>on</strong>stantly developing<br />

competitive envir<strong>on</strong>ment and to reach informati<strong>on</strong> required for further decisi<strong>on</strong> making. The smooth<br />

cooperati<strong>on</strong> between the individual departments of the company can be ensured <strong>on</strong>ly by well-adjusted<br />

data (informati<strong>on</strong>) transfer. It is important to pay attenti<strong>on</strong> to the fact that during the use of informati<strong>on</strong><br />

systems the meaning of informati<strong>on</strong> may be limited to simple data and to work with them. The<br />

meaning of informati<strong>on</strong> is the thing that makes it necessary for all managerial activities. To maintain<br />

the role of informati<strong>on</strong>, we need the knowledge and the cooperati<strong>on</strong> of people from different areas,<br />

not <strong>on</strong>ly IT staff and managers. Some disciplines are significantly c<strong>on</strong>nected with informati<strong>on</strong><br />

management, others give it the knowledge <strong>on</strong>ly marginally. Some disciplines are seen as those that<br />

cooperate with informati<strong>on</strong> management actively, others not. If we study this issue more detailed, we<br />

will find out that also the disciplines that are not c<strong>on</strong>nected with informati<strong>on</strong> management have<br />

anything to offer. Maybe, we just used to see all data provided by them as the given unchanging fact,<br />

and that´s why we d<strong>on</strong>´t pay too much attenti<strong>on</strong> to further analysis of this topic.<br />

References<br />

Alexander, F. and Stevens, R. (2002) Writing Better Requirements, U.K.: Pears<strong>on</strong>.<br />

Blaha, Z. S. (2004) Řízení rizika a finanční inženýrství, Praha: <strong>Management</strong> Press.<br />

Blažek, V. (1997) ‘Informační systém pro řízení podniku (Problémy implementace)’, Moderní řízení, pp. 53-57.<br />

Boehm, V. (1998) ‘Informační zdroje’, Českomoravský profit, vol. 9, pp. 5.<br />

Brož, I. (1997) ‘<strong>Management</strong> na rozcestí’, Moderní řízení, pp. 4-5.<br />

Cejpek, J. (2005) Informace, Komunikace a Myšlení, Praha: Karolinum.<br />

Čechová, A. (1997) „Manažeři budou v informační společnosti měnit své myšlení“, [<strong>on</strong>line], FIRST Innovati<strong>on</strong><br />

Park, www.park.cz/manazeri_budou_v_informacni_spolecnosti_menit_sve_mysleni/ (accessed <strong>on</strong><br />

07/04/97).<br />

CzechInvest. (2012) „Definice malého a středního podnikatele“, [<strong>on</strong>line], CzechInvest,<br />

www.czechinvest.org/definice-msp (accessed <strong>on</strong> 16/05/2012).<br />

Čermák, M. (2007) Řízení informačních rizik v praxi, Praha:Draft.<br />

Doctorandus. (2012) „Vědecké metody ve společenských vědách“, [<strong>on</strong>line], Doctorandus,<br />

www.doctorandus.info/info/e_kapitoly/vedecke_metody.doc (accessed <strong>on</strong> 15/05/12).<br />

Doucek, P. (2010) Informační <strong>Management</strong>, Praha: Professi<strong>on</strong>al Publishing.<br />

Doucek, P. (2004) Řízení projektů informačních systémů. Praha: Mařík.<br />

Garlick, A. (2007) Estimating Risk. A <strong>Management</strong> Approach, Hampshire: Gower.<br />

Hull, E. and Jacks<strong>on</strong>, K. and Dick, J. (2011) Requirements Engineering, L<strong>on</strong>d<strong>on</strong>: Springer.<br />

Imler, K. (2008) Strategické systémy kvality. Pardubice: Lévay.<br />

Jersáková, J. (2012) „Literární rešerše“, [<strong>on</strong>line], JCU, http:// kbe.prf.jcu.cz/files/studenti/Literarni_reserse.pdf<br />

(accessed <strong>on</strong> 15/05/12).<br />

Jirásek, J. (2008) <strong>Management</strong> Budoucnosti, Praha: Professi<strong>on</strong>al Publishing.<br />

Kruliš, J. (2011) Jak Vítězit nad Riziky. Aktivní <strong>Management</strong> rizik – Nástroj Řízení Úspěšných Firem, Praha:<br />

Linde.<br />

381


Marie Černá<br />

Laplante, P. A. (2009) Requirements Engineering for Software and Systems (Applied Software Engineering<br />

Series), Auerbach Publicati<strong>on</strong>s.<br />

Leffingwell, D. and Widrig, D. (2003) Managing Software Requirements. A Use Case Approach, 2 nd editi<strong>on</strong>, U.K.:<br />

Wesley.<br />

Merna, T. and Faisal, F. A. (2007) Risk <strong>Management</strong> (Řízení Rizika ve Firmě), Brno: Computer Press.<br />

Mládková, L. (2004) <strong>Management</strong> Znalostí v Praxi, Praha: Professi<strong>on</strong>al Publishing.<br />

Oracle. (2011) „Internal Auditing´s Role in Risk <strong>Management</strong>“, [<strong>on</strong>line], Oracle,<br />

www.oracle.com/us/soluti<strong>on</strong>s/corporate-governance/ia-role-in-rm-345774.pdf (accessed <strong>on</strong> 15/05/12).<br />

Ortiz, L. V. (2004) „Risk <strong>Management</strong>“, [<strong>on</strong>line], Adbi,<br />

www.adbi.org/files/2004.04.12.cpp.risk.management.accounting.pdf (accessed <strong>on</strong> 14/05/12).<br />

Petříková, R. (2010) Moderní <strong>Management</strong> Znalostí, Praha: Professi<strong>on</strong>al Publishing.<br />

Prokůpková, D. (2007) „Řízení rizik v orgánech veřejné správy“, [<strong>on</strong>line], Bilance, www.bilance.cz (accessed <strong>on</strong><br />

12/02/12).<br />

Smejkal, V. and Rais, K. (2006) Řízení rizik ve firmách a jiných organizacích. Praha: Grada.<br />

Sokolowsky, P. (2002) Informační Požadavky Moderního Podniku 1, Praha: Karolinum.<br />

Sokolowsky, P. (2002) Organizace a <strong>Management</strong> Podnikového Zpracování Informací 2, Praha: Karolinum.<br />

St<strong>on</strong>ebumer, G., Goguen, A. and Feringa, A. (2002) Risk <strong>Management</strong> Guide for Informati<strong>on</strong> Technology<br />

Systems, Gaithersburg: NIST Special Publicati<strong>on</strong>.<br />

Stýblo, J. (2008) <strong>Management</strong> Současný a Budoucí, Praha: Trit<strong>on</strong>.<br />

Svatá, V. (2011) Audit Informačního Systému, Praha: Professi<strong>on</strong>al Publishing.<br />

Toth, P. (1993) Informační Systémy Státní Správy a Územní Samosprávy, Praha: VŠE.<br />

Vodáček, L. and Rosický, A. (1997) Informační <strong>Management</strong>, Praha, <strong>Management</strong> Press.<br />

Wiegers, E. K. (2008) Požadavky na Software. Od Zadání k Architektuře Aplikace, Brno: Computer Press.<br />

382


A Methodology for Competitive Intelligence Metrics<br />

Rhiann<strong>on</strong> Gainor and France Bouthillier<br />

School of Informati<strong>on</strong> Studies, McGill University, M<strong>on</strong>treal, Canada<br />

rhiann<strong>on</strong>.gainor@mail.mcgill.ca<br />

france.bouthillier@mcgill.ca<br />

Abstract: The literature <strong>on</strong> competitive intelligence (CI) reveals that a significant challenge exists in measuring<br />

the outcomes and impact of intelligence. There are process measures in use, but little attenti<strong>on</strong> has been given to<br />

measure for outcomes and impact. This imbalance has been attributed to methodological and c<strong>on</strong>ceptual<br />

problems of measurement. This paper proposes a case study methodology by which a model may be developed<br />

for measuring the relati<strong>on</strong>ship between CI products and organizati<strong>on</strong>al outcomes. It examines a given decisi<strong>on</strong>,<br />

the intelligence products that informed it, and links those products through the decisi<strong>on</strong> to the outcomes, and their<br />

impact up<strong>on</strong> the organizati<strong>on</strong>. The methodology combines subjective assessments made by decisi<strong>on</strong> makers and<br />

other employees of past decisi<strong>on</strong>s and their outcomes with organizati<strong>on</strong>al data obtained through document<br />

analysis, to compare expert opini<strong>on</strong> to objective data. This triangulati<strong>on</strong> of data seeks to link outcomes with three<br />

indicators (financial outputs, innovati<strong>on</strong>, and client relati<strong>on</strong>ships) to an organizati<strong>on</strong>’s strategic plan to assess<br />

impact of CI.<br />

Keywords: informati<strong>on</strong> management; decisi<strong>on</strong> making; intangibles measurement; competitive intelligence;<br />

impact measures<br />

1. Introducti<strong>on</strong><br />

Competitive intelligence (CI) is both a process and a product. It is the process by which an<br />

organizati<strong>on</strong>, or an individual, takes informati<strong>on</strong> and analyzes it, to understand the competitive<br />

envir<strong>on</strong>ment. It is also the products that result from such analysis, such as reports and company<br />

profiles, which are then used to inform decisi<strong>on</strong> making. CI m<strong>on</strong>itors and attempts to anticipate the<br />

competitive envir<strong>on</strong>ment for competitive advantage, and is popularly believed to provide organizati<strong>on</strong>s<br />

with a competitive edge. There is very little research proving a causal relati<strong>on</strong>ship where CI produces<br />

organizati<strong>on</strong>al success, however, chiefly because CI suffers as a field of research and practice from<br />

problematic measurement. As will be shown, although the literature states that there is a need for<br />

measurement, there is little measurement in practice, due to inc<strong>on</strong>sistent c<strong>on</strong>ceptualizati<strong>on</strong>s and<br />

methodological complicati<strong>on</strong>s.<br />

It is necessary to preface this paper with a comment <strong>on</strong> an inc<strong>on</strong>sistent c<strong>on</strong>ceptualizati<strong>on</strong> of the CI<br />

field. The literature review in this paper draws up<strong>on</strong> research in business intelligence (BI) and<br />

competitive intelligence activities. Bouthillier and Shearer (2003), in their review of BI and CI, note that<br />

although in some instances the terms have been used interchangeably, the scope of BI is typically<br />

larger, including internal and external informati<strong>on</strong> activities for the organizati<strong>on</strong>, while CI is narrower in<br />

scope, focusing solely <strong>on</strong> the competitive external envir<strong>on</strong>ment. As described the c<strong>on</strong>ceptual review<br />

d<strong>on</strong>e by Buchda (2007) the terms BI and CI share comm<strong>on</strong> processes (sourcing data for analysis) for<br />

similar purposes (to inform decisi<strong>on</strong> making, and support management). For this paper, ‘CI’ will be<br />

c<strong>on</strong>sidered a syn<strong>on</strong>ym for BI, recognizing those comm<strong>on</strong>alities.<br />

The purpose of this paper is to propose a case study methodology by which a model may be<br />

developed for measuring the relati<strong>on</strong>ship between CI products and organizati<strong>on</strong>al outcomes. This<br />

methodology has been developed in resp<strong>on</strong>se to discussi<strong>on</strong>s in the CI literature, as part of a doctoral<br />

research project. The proposed methodology attempts to address some of the oft-cited challenges to<br />

developing useful and accurate measures of CI outcomes and impact for organizati<strong>on</strong>s.<br />

The paper c<strong>on</strong>tains the following elements: a brief review of proposed approaches and methods of<br />

measurement in the literature found during searches of the databases LISA, LISTA, and Library and<br />

Informati<strong>on</strong> Studies Full Text from September 2010 to October 2011, using terms such as “measure*<br />

AND competitive intelligence”; a discussi<strong>on</strong> of the c<strong>on</strong>ceptual issues involved, including presentati<strong>on</strong><br />

of the c<strong>on</strong>ceptual framework for the proposed research methodology; a descripti<strong>on</strong> of the proposed<br />

methodology, relating it to issues identified in the literature; identificati<strong>on</strong> of the value and originality of<br />

the proposed research method; and some c<strong>on</strong>cluding remarks.<br />

383


Rhiann<strong>on</strong> Gainor and France Bouthillier<br />

2. A review of models of measurement in the literature<br />

The results of CI practiti<strong>on</strong>er surveys have indicated that organizati<strong>on</strong>s tend to do little, if any, formal<br />

measurement of CI processes, products, or outcomes (Herring, 1996; Marin and Poulter, 2004;<br />

Prescott and Bharwaj, 1995). It is unsurprising then, that little has been written about CI performance<br />

assessment. Blenkhorn and Fleisher (2007) point out in the literature review prefacing their study of<br />

CI practiti<strong>on</strong>ers that most of what has been written about CI assessment c<strong>on</strong>sists of practiti<strong>on</strong>ers’<br />

anecdotal experiences rather than scholarly (meaning rigorous, valid, and reliable) studies. The<br />

measurement methods and tools reviewed here are prescriptive, proposed by scholars, and have not<br />

been subject to testing, although calls have been made for more empirical testing of CI measurement<br />

methods (e.g., Pirttimäki, Lönnqvist, and Karjaluoto, 2006).<br />

Reviewing measures that have been proposed in the CI literature, Buchda (2007) proposed a<br />

classificati<strong>on</strong> and analysis framework for them, grouping them into three types:<br />

Measures of Effectiveness (MOE)<br />

Return <strong>on</strong> Investment (ROI)<br />

Balanced Scorecard-Related (BSC-Related)<br />

In this secti<strong>on</strong> each of these types will be summarized and critiqued for its applicati<strong>on</strong> to CI c<strong>on</strong>texts.<br />

2.1 Measures of effectiveness (MOE)<br />

The MOE approach was advocated by Herring (1996) at the c<strong>on</strong>clusi<strong>on</strong> of multi-stage exploratory<br />

research he c<strong>on</strong>ducted in the 1990s for the Society of Competitive Intelligence Professi<strong>on</strong>als (SCIP),<br />

to investigate possible methods of performance measurement for CI. MOEs, simply put, identify<br />

outcomes that indicate CI did its job, with the intent of proving functi<strong>on</strong>ality and value to justify<br />

investment. Herring suggests four MOEs: time savings, cost savings, cost avoidance, and revenue<br />

enhancement.<br />

In another example, a set of potential MOEs is provided in a study by The Futures Group (in Davis<strong>on</strong>,<br />

2001; Herring, 1996; McG<strong>on</strong>agle and Vella, 2002). The study c<strong>on</strong>sisted of interviews with US<br />

companies to identify their MOEs. The most comm<strong>on</strong> measures identified were:<br />

Acti<strong>on</strong>s taken<br />

Market share changes<br />

Financial goals met<br />

Leads generated<br />

New products developed<br />

Herring (1996) supported the use of MOEs because he c<strong>on</strong>cluded that measurement of CI<br />

effectiveness (effectiveness being how well CI is achieving its goals, see Fleisher and Blenkorn, 2001)<br />

required a qualitative and quantitative evaluati<strong>on</strong> of the entire CI cycle in cooperati<strong>on</strong> with<br />

management, in order to ensure that CI activities were aligned with strategic objectives, thus ensuring<br />

value. He argues that an MOE approach allows the user to address management expectati<strong>on</strong>s, and<br />

establishes a framework for the most valuable assessment of CI: that of the executives using and<br />

overseeing the CI processes and products. The problem with using MOEs, however, is that<br />

implementing them can be difficult, and as Buchda (2007) pointed out, MOEs typically are selected <strong>on</strong><br />

the basis of desirability, rather than research that has proven what benefits can be expected to result<br />

from CI.<br />

2.2 Return <strong>on</strong> Investment (ROI)<br />

ROI methods attempt to prove profit by subtracting costs from the revenue generated by a given<br />

activity. Davis<strong>on</strong> (2000) brought together literature <strong>on</strong> CI and literature <strong>on</strong> advertising effectiveness<br />

measures to c<strong>on</strong>ceptualize a Competitive Intelligence Measurement Model (CIMM). This model was<br />

developed to provide a more tangible method than MOEs for determining CI effectiveness. The<br />

formula below gives the return <strong>on</strong> investment for competitive intelligence (ROCII), with the intent to<br />

produce an answer quantifiable in dollars:<br />

ROCII = (CI outputs – CI inputs) / CI inputs<br />

384


Rhiann<strong>on</strong> Gainor and France Bouthillier<br />

Kilmetz and Bridge (1999) provide a practiti<strong>on</strong>ers’ report of how to use ROI methods, recounting a<br />

business case, to illustrate the need for users of ROI methods to engage in modelling potential<br />

scenario outcomes of current decisi<strong>on</strong>s. These potential scenarios are developed in order to forecast<br />

likely and hoped-for returns, a task that provides data in the form of most-likely outputs, to be used in<br />

formulas such as the <strong>on</strong>e above.<br />

Davis<strong>on</strong> (2000) acknowledges three significant weaknesses of a ROI method. These weaknesses are<br />

uncertainty regarding the accuracy of forecasts used in the model, the inability of the formula to<br />

account for intangible results and qualities of CI, and that the model does not include any<br />

c<strong>on</strong>siderati<strong>on</strong> of organizati<strong>on</strong>al strategy. Davis<strong>on</strong> suggests possible soluti<strong>on</strong>s for two of these<br />

weaknesses. First, to gauge the potential accuracy of current predicti<strong>on</strong>s by evaluating the accuracy<br />

of previous predicti<strong>on</strong>s. Sec<strong>on</strong>d, to use Likert scales to evaluate decisi<strong>on</strong> makers’ satisfacti<strong>on</strong><br />

regarding the intangible qualities of CI products and processes, such as quality, relevance, accuracy,<br />

etc. For the third weakness, however, Davis<strong>on</strong> provides no soluti<strong>on</strong> or proposed additi<strong>on</strong>al measure,<br />

simply stating that a ROI model, not having l<strong>on</strong>g-term data, cannot measure strategic outputs and<br />

outcomes.<br />

2.3 Balanced scorecard (BSC)<br />

The balanced scorecard, originally developed by Kaplan and Nort<strong>on</strong> (1992), is a measurement tool<br />

that allows users to examine organizati<strong>on</strong>al functi<strong>on</strong>s from multiple perspectives, in relati<strong>on</strong> to <strong>on</strong>e<br />

another, with the intent of m<strong>on</strong>itoring and improving performance. Lönnqvist and Pirttimäki (2006), in<br />

their literature review to identify and assess measurement approaches for determining value and<br />

managing processes for CI in an organizati<strong>on</strong>, suggest that a BSC approach is the most beneficial.<br />

Pirttimäki, Lönnqvist, and Karjaluoto (2006) undertook a case study, applying a BSC to the CI unit of a<br />

company, and argued that this approach has to be tailored to the needs of a specific c<strong>on</strong>text and<br />

organizati<strong>on</strong>, including tracking of usage rates for CI products, satisfacti<strong>on</strong> surveys, and win/loss<br />

ratios for specific decisi<strong>on</strong>s.<br />

While the BSC allows the user to address some of the complexity of CI processes, its weakness is<br />

that it does not show clear evidence of causal relati<strong>on</strong>ships between CI inputs and outcomes<br />

(Buchda, 2007; Nørreklit, 2000).<br />

All of the approaches named in this secti<strong>on</strong> are useful in providing to the user some quantificati<strong>on</strong> of<br />

CI performance. However, as described above, the weakness of these measurement approaches is<br />

that they do not provide evidence of causal relati<strong>on</strong>ships, which is also a problem for adapting<br />

intangibles measures in the fields of knowledge management (KM) and intellectual capital (IC) to CI,<br />

for example Alpha IC and Skandia Navigator. Although these measures are useful for identifying and<br />

representing intangible assets, they do not provide evidence of dynamic causal relati<strong>on</strong>ships. These<br />

relati<strong>on</strong>ships are key to measuring the value of CI through its outcomes and impacts, as will be<br />

argued below.<br />

3. Measurement of CI in practice<br />

Prescott and Bharwaj (1995) did a large-scale survey of members of the Society of Competitive<br />

Intelligence Professi<strong>on</strong>als (SCIP) to understand the comp<strong>on</strong>ents of CI programs. Resp<strong>on</strong>dents<br />

indicated that although they believed that CI benefits could be seen in decisi<strong>on</strong> making, sharing<br />

informati<strong>on</strong>, and identifying new opportunities, they were uncertain as to how CI impacted strategic<br />

areas in their organizati<strong>on</strong>s, namely market positi<strong>on</strong>, revenues, customer service, and increased<br />

capabilities. The authors suggested that metrics needed to be developed to enable CI units to better<br />

assess their role and impact within organizati<strong>on</strong>s.<br />

Nearly a decade later, Marin and Poulter (2004) did another survey of SCIP members, with some<br />

interviews of survey participants. The purpose of their study was to better understand CI practices and<br />

practiti<strong>on</strong>ers. One result of the study was that “few organizati<strong>on</strong>s have any mechanisms in place to<br />

measure the value of competitive intelligence,” though some organizati<strong>on</strong>s made an attempt to track<br />

usage of electr<strong>on</strong>ic CI resources (2004: 172). The authors attributed, at least in part, the n<strong>on</strong>existence<br />

of measures to the problem of quantifying CI value and effectiveness.<br />

Studies such as these have repeatedly found that organizati<strong>on</strong>s using CI are not measuring CI<br />

processes or outcomes, although there are documented excepti<strong>on</strong>s (e.g., Pirttimäki, Lönnqvist, and<br />

385


Rhiann<strong>on</strong> Gainor and France Bouthillier<br />

Karjaluoto 2006). Herring (1996) c<strong>on</strong>ducted a small field survey and found that, as did Marin and<br />

Poulter (2004), organizati<strong>on</strong>s using CI do almost no evaluati<strong>on</strong> or measurement of CI. N<strong>on</strong>e of the<br />

executives he surveyed were using any formal evaluati<strong>on</strong>, although they might do unexpected and<br />

informal evaluati<strong>on</strong> when reviewing budgets and/or trying to c<strong>on</strong>trol costs. Blenkhorn and Fleisher<br />

(2007) c<strong>on</strong>firmed there are few formal measures in use, or often no measures at all.<br />

Authors reporting <strong>on</strong> measurement repeatedly call for measures to be developed, recognizing that<br />

they are necessary to c<strong>on</strong>ceptually develop the field, and strengthen research. Practiti<strong>on</strong>ers of CI,<br />

when surveyed, have stated that they are aware of the need for measurement and c<strong>on</strong>sider its<br />

development a priority for their field of practice (Hannula and Pirttimaki, 2003; Qingjiu and Prescott,<br />

2000).<br />

4. Why is measuring CI so difficult?<br />

One problem with CI measurement is related to the difficulty of c<strong>on</strong>ceptualizing CI value in order to<br />

distinguish between effectiveness, results or benefits, outputs, outcomes, and impact. Another<br />

problem is isolating CI processes and products. Methodological issues include identifying intangibles<br />

related to CI, and relating results to ‘acti<strong>on</strong>able’ CI. Some of these c<strong>on</strong>ceptual and methodological<br />

issues are briefly discussed here in the c<strong>on</strong>text of studies d<strong>on</strong>e <strong>on</strong> CI measurement.<br />

Wright and Calof (2006) examined three empirical studies carried out in Canada, the UK and Europe,<br />

to draw comparis<strong>on</strong>s of their respective approaches and findings. In reviewing their study’s findings,<br />

the authors commented that<br />

…there is little c<strong>on</strong>sistency in terms of measurement and output value. In addressing the<br />

critical area of intelligence, there has to be some agreement within the field <strong>on</strong> how to<br />

operati<strong>on</strong>alise the intelligence c<strong>on</strong>struct. While the definiti<strong>on</strong> is generally accepted,<br />

measurement is not (2006: 462).<br />

Calof and Wright (2008) found in their bibliometric assessment of CI literature that part of the problem<br />

with c<strong>on</strong>ceptualizati<strong>on</strong> of measurement lay in studies attempting to measure the entire CI model, both<br />

processes and products, rather than closely examining and testing elements of the model and<br />

attempting to link CI to the performance measures of the organizati<strong>on</strong>. As a result, they recommend<br />

that measurement studies, rather than attempting to focus <strong>on</strong> the entirety of CI functi<strong>on</strong>s, roles, and<br />

products, should instead examine smaller comp<strong>on</strong>ents of CI to improve the validity of the research.<br />

As part of this effort to more rigorously examine CI, it is necessary to first, more clearly c<strong>on</strong>ceptualize<br />

what the c<strong>on</strong>structs of the field are. What, for example, is the purpose of CI? What are the benefits of<br />

CI?<br />

As discussed earlier in the secti<strong>on</strong> <strong>on</strong> measurement methods and approaches in the literature, CI<br />

benefits can be identified as indicators of effectiveness, such as time savings. Hannula and Pirttimaki<br />

(2003) found that for Finnish companies, their most-expected benefits of CI were better informati<strong>on</strong> for<br />

decisi<strong>on</strong> making; ability to anticipate threats and opportunities; growing knowledge; and savings of<br />

time and m<strong>on</strong>ey. Marin and Poulter (2004) who studied CI practiti<strong>on</strong>ers discovered that CI often is<br />

intended to help decisi<strong>on</strong> makers to make decisi<strong>on</strong>s. Jaworski and Wee (1992) found that CI was<br />

designed to help increasing the quality of strategic planning by improving knowledge of the market.<br />

Qingjiu and Prescott, (2000) who studied Chinese CI practiti<strong>on</strong>ers, found that resp<strong>on</strong>dents believed<br />

that CI should result in improvements for decisi<strong>on</strong> making and customer service.<br />

Of all lists of reas<strong>on</strong>s given by researchers, providing help to decisi<strong>on</strong> making - often described as<br />

strategic decisi<strong>on</strong> making - is the most comm<strong>on</strong>ly cited reas<strong>on</strong> for implementing CI (see also Herring,<br />

1996; Bose, 2008). It is our c<strong>on</strong>tenti<strong>on</strong> that most indicators of this improved decisi<strong>on</strong> making, as<br />

described in the literature reviewed in the previous paragraph, can be loosely grouped under three<br />

categories: financial outputs, improved client relati<strong>on</strong>ships, and innovati<strong>on</strong> in products and services.<br />

Since supporting, and presumably improving, decisi<strong>on</strong> making is the most comm<strong>on</strong>ly-cited reas<strong>on</strong> for<br />

CI to be developed and used in organizati<strong>on</strong>s; this presents a challenge for developing measures.<br />

Unlike process measures which can, for example, calculate employee-hours against numbers of CI<br />

products developed, measures attempting to c<strong>on</strong>nect CI practices and products to decisi<strong>on</strong> making<br />

cannot be as straightforward. Decisi<strong>on</strong> making is not a linear process, and it can be complex. Any tool<br />

that measures decisi<strong>on</strong> making and its effects will necessarily involve qualitative methods. Further<br />

complicati<strong>on</strong>s are related to questi<strong>on</strong>s of accuracy in self-reporting <strong>on</strong> what is essentially an internal<br />

386


Rhiann<strong>on</strong> Gainor and France Bouthillier<br />

and subjective activity, and in allowing time to lapse for intangibles to appear (Kujansivu and<br />

Lönnqvist, 2009). Although no measurement model has yet been developed that addresses CI’s role<br />

in decisi<strong>on</strong> making, there has been a call by a practiti<strong>on</strong>er to examine how CI factors into decisi<strong>on</strong><br />

making (Sawka, in Blenkhorn and Fleisher, 2007; Lönnqvist and Pirttimäki 2006; Marin and Poulter,<br />

2004) which has been echoed by scholars’ comments.<br />

5. C<strong>on</strong>ceptualizing a revised model of measurement<br />

CI outputs and outcomes are not clearly distinguished in the literature. In both the articles by Buchda<br />

(2007) and Davis<strong>on</strong> (2000), outputs are described as both the tangible and immediately visible<br />

results, such as use of CI products, and the intangible l<strong>on</strong>g-term results such as fulfilment of an<br />

objective. In their case study, Pirttimäki, Lönnqvist, and Karjaluoto (2006) more clearly distinguish<br />

outputs from other results of CI, describing outputs as assignments completed and user satisfacti<strong>on</strong>,<br />

which produce in turn intangible “effects” such as improved decisi<strong>on</strong> making, that may then lead to<br />

financial c<strong>on</strong>sequences for an organizati<strong>on</strong>. In the model proposed by this paper, outputs and<br />

outcomes are distinguished from <strong>on</strong>e another, borrowing a definiti<strong>on</strong> of outcomes from Boyce,<br />

Meadow, and Kraft’s text <strong>on</strong> measurement in the informati<strong>on</strong> sciences: “Outcomes are the results of a<br />

system’s operati<strong>on</strong>s. Desirable outcomes are really the broad goals or objectives for which the system<br />

was created...Outcomes are generally not tangible” (1994: 242). For this model, outcomes build <strong>on</strong><br />

outputs, outputs being the tangible and immediately observable results of CI use in the decisi<strong>on</strong>making<br />

process.<br />

The literature frequently states, when describing the problems of CI measurement, that industries and<br />

organizati<strong>on</strong>s are too diverse to permit the creati<strong>on</strong> of <strong>on</strong>e standard measure (e.g., Kilmetz and<br />

Bridge, 1999; Lönnqvist and Pirttimäki, 2006; Rothberg and Ericks<strong>on</strong>, 2005). The research<br />

methodology proposed by this paper suggests that if we c<strong>on</strong>ceptualize the primary value of CI as<br />

being improved strategic decisi<strong>on</strong> making, and select indicators of decisi<strong>on</strong> making effectiveness that<br />

address both the tangible immediate outputs of a decisi<strong>on</strong>, and the l<strong>on</strong>ger-term intangible outputs of<br />

the decisi<strong>on</strong>, a basic, more generic model of measurement could be developed that would be<br />

applicable across multiple organizati<strong>on</strong>s and industries.<br />

Given the issues raised in the literature, we suggest that a measure is needed that places an<br />

emphasis <strong>on</strong> effectiveness, meaning how well CI is meeting its goal is in making positive c<strong>on</strong>tributi<strong>on</strong>s<br />

to decisi<strong>on</strong> making. Because this proposed measurement model would focus <strong>on</strong>ly <strong>on</strong> effectiveness,<br />

rather than CI as a process or product, it would follow the recommendati<strong>on</strong>s made by Calof and<br />

Wright (2008) to build measurement models by examining smaller comp<strong>on</strong>ents, rather than<br />

attempting to measure all of CI. Instead it would examine <strong>on</strong>ly the outcomes of a decisi<strong>on</strong> influenced<br />

by CI, to determine whether CI truly improved the decisi<strong>on</strong>-making process – a benefit all authors<br />

agree should be the result of CI use. By focusing <strong>on</strong> strategic decisi<strong>on</strong> making, and tying the<br />

outcomes of decisi<strong>on</strong>s to the larger strategic plans of the organizati<strong>on</strong>, it would also attempt to<br />

measure the organizati<strong>on</strong>al impact of CI (Kujansivu and Lönnqvist, 2009; Poll and Payne, 2006). In<br />

our c<strong>on</strong>ceptual model, CI is viewed as <strong>on</strong>e several inputs factoring into a decisi<strong>on</strong>. Building <strong>on</strong> a<br />

three-stage model of decisi<strong>on</strong> making developed by Nicholas (2004) during his four-year study of<br />

decisi<strong>on</strong> making in 92 firms, the figure below has been developed. Please note that Nicholas (2004)<br />

heavily based his model <strong>on</strong> Herbert Sim<strong>on</strong>’s c<strong>on</strong>cept of organizati<strong>on</strong>al decisi<strong>on</strong> making. In the model,<br />

a problem is identified. Various inputs, including CI, go into identifying a range of opti<strong>on</strong>s to be acted<br />

up<strong>on</strong>, and the choice perceived at the time to be the most optimal is chosen. From the decisi<strong>on</strong>,<br />

immediate and tangible outputs appear. As time goes <strong>on</strong>, intangible outcomes based in the outputs,<br />

manifest, and when related to an organizati<strong>on</strong>’s strategic plan, dem<strong>on</strong>strate impact of the decisi<strong>on</strong>.<br />

The figure below provides a visual:<br />

Figure 1: CI as an input into the decisi<strong>on</strong>-making process<br />

387


Rhiann<strong>on</strong> Gainor and France Bouthillier<br />

In the proposed model, a ‘good’ measure becomes <strong>on</strong>e that allows time for impact to manifest,<br />

accounts for the role of CI in decisi<strong>on</strong> making, and relates CI to organizati<strong>on</strong>al strategy. Outcome<br />

indicators of value or benefit could then be traced through tangible outputs in multiple dimensi<strong>on</strong>s of<br />

the organizati<strong>on</strong>. Some examples are given in the following table:<br />

Table 1: Outcomes and outputs of a decisi<strong>on</strong><br />

Sample outputs Outcomes<br />

Cost savings<br />

Time savings<br />

Improved financial outputs<br />

Revenue generated<br />

Increased CI requests by internal clients<br />

External client retenti<strong>on</strong><br />

Improved client relati<strong>on</strong>ships<br />

Increased sales<br />

New products<br />

Increased innovati<strong>on</strong><br />

New services<br />

Such a c<strong>on</strong>ceptual model of outputs and outcomes would then allow the scholar to obtain the<br />

elements needed for the ‘good’, multi-dimensi<strong>on</strong>al CI measurement as suggested by Blenkhorn and<br />

Fleisher (2007), for which tangible/intangible, qualitative/quantitative, subjective/objective data can be<br />

collected.<br />

6. Proposed methodology<br />

Wright and Calof (2006) and Marin and Poulter (2004) recommended case study research and direct<br />

observati<strong>on</strong> of CI practices to provide empirical data that can inform the development of a CI<br />

measurement model. The research methodology proposed here is a case study of an organizati<strong>on</strong>,<br />

either a government agency or public company, with a CI unit that employs between 2 and 10<br />

dedicated full-time CI employees, to ensure it has a significant amount of resources dedicated to CI,<br />

and yet, for practical reas<strong>on</strong>s, is not too large. The CI unit will have been in operati<strong>on</strong> for at least 5<br />

years, to assure maturity in its operati<strong>on</strong>s, and has had a strategic plan detailing goals for the<br />

organizati<strong>on</strong> in place for more than 3 years.<br />

The choice of a public company or government agency means that much of the informati<strong>on</strong> sought,<br />

such as market share and profitability, is within the public domain, and the organizati<strong>on</strong> will hopefully<br />

be more open to sharing related informati<strong>on</strong> than a strictly private firm might be.<br />

This case study design owes much to Dalkir and McIntyre (2011) and their suggested result-based<br />

management accountability framework (RMAF) approach to intelligence evaluati<strong>on</strong> research,<br />

specifically how they aligned their measure with assets of strategic importance to the organizati<strong>on</strong>,<br />

and used indicators specific to the initiative to collect quantitative, qualitative, and anecdotal data. The<br />

emphasis of this research is slightly different however, in that a goal of the research is to prove the<br />

value of <strong>on</strong>e specific factor (CI) in the decisi<strong>on</strong>s made.<br />

The research methodology calls for the use of two data collecti<strong>on</strong> methods: interviews and document<br />

analysis. Interviews will be used with decisi<strong>on</strong> makers to identify and analyze the role of CI in several<br />

significant strategic decisi<strong>on</strong>s made in the past 3-5 years that were informed by CI products. This<br />

timeframe was chosen to try and balance the need for time, for results to appear from a decisi<strong>on</strong>, yet<br />

not be so far distant that those involved with the decisi<strong>on</strong> would be impossible to interview. This<br />

retrospective decisi<strong>on</strong> analysis hopefully will circumvent <strong>on</strong>e of the obstacles to measuring CI<br />

outcomes and impact, namely time. Interviews will also be held with CI unit employees, for the<br />

researcher to understand what performance measures they may already use, and what organizati<strong>on</strong>al<br />

processes and usage exist around CI in the organizati<strong>on</strong>. Both groups, decisi<strong>on</strong> makers and CI unit<br />

employees, will be asked for their subjective evaluati<strong>on</strong>s of the strategic decisi<strong>on</strong>s identified, and what<br />

outcomes and impacts they attribute to those decisi<strong>on</strong>s.<br />

Data will also be collected through document analysis, triangulating subjective opini<strong>on</strong>s of the study<br />

subjects with references to company documents such as industry analysis reports, meeting notes,<br />

press releases, and the organizati<strong>on</strong>’s strategic plan. Review of these documents will provide some<br />

more objective data, possibly clarifying some of the difficulties in retrospective analysis of decisi<strong>on</strong><br />

making.<br />

388


Rhiann<strong>on</strong> Gainor and France Bouthillier<br />

Once this data has been collected, the researcher will do some preliminary data analysis and<br />

develop:<br />

Influence diagrams representing influences at play for each decisi<strong>on</strong> made (Diffenbach, 1982);<br />

Brief sketches of the organizati<strong>on</strong>’s competitive envir<strong>on</strong>ment at the time of each decisi<strong>on</strong> and at<br />

the time of the study (“snapshots” as described by Dumay, 2009);<br />

Visual models of organizati<strong>on</strong>al processes around CI unit use and products;<br />

Visual models of organizati<strong>on</strong>al supports for decisi<strong>on</strong>-making used for each of the decisi<strong>on</strong>s; and<br />

A summary of the subjective evaluati<strong>on</strong>s given by each participant of the outcomes and impacts<br />

of the decisi<strong>on</strong>(s) with which s/he was involved.<br />

These documents will then be taken back to study participants. Each study participant will be shown<br />

the documents relating to his or her interviews and asked to check the accuracy of the researcher’s<br />

understanding of their subjective evaluati<strong>on</strong>s (Yin, 1994). Data will then be analysed to try and<br />

discover first, what weight CI had in each strategic decisi<strong>on</strong>; and sec<strong>on</strong>d, outcomes related to<br />

innovati<strong>on</strong>, relati<strong>on</strong>ships, financial outputs. The interviews and the outcomes will then be related to<br />

the organizati<strong>on</strong>’s strategic plan (both past and current), to identify impact, answering the questi<strong>on</strong>:<br />

did CI help the organizati<strong>on</strong> make a decisi<strong>on</strong> that successfully accomplished some element of the<br />

strategic plan?<br />

7. Value/originality of approach<br />

The value of this proposed c<strong>on</strong>ceptual framework and research methodology is threefold. It goes to<br />

the heart of CI’s role in advising/influencing a decisi<strong>on</strong>, which other measurement methods and<br />

recommendati<strong>on</strong>s do not currently examine as an activity subject to a variety of influences. By<br />

developing a measurement model of outcomes and impact, with ‘baseline’ or comm<strong>on</strong>ly agreed-up<strong>on</strong><br />

indicators of value and benefit (financial outputs, client relati<strong>on</strong>ships, and innovati<strong>on</strong>), it takes a step<br />

toward developing a model that is applicable to more than <strong>on</strong>e organizati<strong>on</strong>. It also provides<br />

opportunity to discover if outcomes and impacts of CI can actually be measured.<br />

8. C<strong>on</strong>clusi<strong>on</strong><br />

This paper presents a research methodology that could potentially address some of the c<strong>on</strong>ceptual<br />

and methodological issues which have historically challenged the development of CI measures. To<br />

that end, this paper has reviewed some of the measurement approaches and tools presented in CI<br />

literature, summarizing some of the most pertinent discussi<strong>on</strong> around problems of CI measurement<br />

and c<strong>on</strong>cepti<strong>on</strong>s of value and benefits. This review has hopefully made evident some of the<br />

justificati<strong>on</strong> and rati<strong>on</strong>ale for the proposed research methodology.<br />

References<br />

Blenkhorn, D.L. and Fleisher, C.S. (2007) “Performance Assessment in Competitive Intelligence: An Explorati<strong>on</strong>,<br />

Synthesis, and Research Agenda”, Journal of Competitive Intelligence and <strong>Management</strong>, Vol. 4, No. 2, pp<br />

4-22.<br />

Bose, R. (2008) “Competitive Intelligence Process and Tools for Intelligence Analysis”, Industrial <strong>Management</strong><br />

and Data Systems, Vol. 108, No. 4, pp 510-528.<br />

Boyce, B. R., Meadow, C. T., & Kraft, D. H. (1994) Measurement in Informati<strong>on</strong> Science. San Diego, CA:<br />

<strong>Academic</strong> Press.<br />

Buchda, S. (2007) “Rulers for Business Intelligence and Competitive Intelligence: An Overview and Evaluati<strong>on</strong> of<br />

Measurement Approaches”, Journal of Competitive Intelligence and <strong>Management</strong>, Vol. 4, No. 2, pp 22-54.<br />

Calof, J. L., and Wright, S. (2008) “Competitive Intelligence: A Practiti<strong>on</strong>er, <strong>Academic</strong>, and Inter-disciplinary<br />

Perspective”, <str<strong>on</strong>g>European</str<strong>on</strong>g> Journal of Marketing, Vol. 42, No. 7/8, pp 717-730.<br />

Dalkir, K., and McIntyre, S. (2011) “Measuring Intangible Assets: Assessing the Impact of Knowledge<br />

<strong>Management</strong> in the S&T Fight against Terrorism,” in B. Vallejo-Al<strong>on</strong>so, A. Rodríguez-Castellanos, and G.<br />

Arregui-Ayastuy, ed.s. Identifying, Measuring, and Valuing Knowledge-Based Intangible Assets: New<br />

Perspective. Hershey, PA: Business Science Reference, pp 156-176.<br />

Davis<strong>on</strong>, L. (2000) “Measuring Competitive Intelligence Effectiveness: Insights from the Advertising Industry”,<br />

Competitive Intelligence Review, Vol. 12, No. 4, pp 25 38.<br />

Diffenbach, J. (1982). “Influence Diagrams for Complex Strategic Issues”, Strategic <strong>Management</strong> Journal, Vol. 3,<br />

No. 2, pp 133-146.<br />

Dumay, J. C. (2009) “Intellectual Capital Measurement: A Critical Approach”, Journal of Intellectual Capital, Vol.<br />

10, No. 2, pp 190-210.<br />

389


Rhiann<strong>on</strong> Gainor and France Bouthillier<br />

Fleisher, C. S. and Blenkhorn, D. L. (2001) Effective Approaches to Assessing Competitive Intelligence<br />

Performance. In C.S. Fleisher and D.L. Blenkhorn, ed. Managing Fr<strong>on</strong>tiers in Competitive Intelligence.<br />

Westport, CT: Quorum Books, pp 110-122.<br />

Hannula, M., and Pirttimaki, V. (2003) “Business Intelligence Empirical Study <strong>on</strong> the Top 50 Finnish Companies”,<br />

American Academy of Business, Vol. 2, No. 2, pp 593-599.<br />

Herring, J. (1996) Measuring the Effectiveness of Competitive Intelligence: Assessing and Communicating CI’s<br />

Value to Your Organizati<strong>on</strong>. Society of Competitive Intelligence Professi<strong>on</strong>als: Alexandria, VA.<br />

Jaworski, B. and Wee, L.C. (1992) “Competitive Intelligence and Bottom-Line Performance”, Competitive<br />

Intelligence Review, Vol. 3, No. 3-4, pp 23-27.<br />

Kaplan, R. S., and Nort<strong>on</strong>, D. P. (1992, January-February) “The Balanced Scorecard: Measures that Drive<br />

Performance”, Harvard Business Review, Vol. 70, No. 1, pp 71-79.<br />

Kilmetz, S.D., and Bridge, A.S. (1999) “Gauging the Returns <strong>on</strong> Investments in Competitive Intelligence”,<br />

Competitive Intelligence Review, Vol. 10, No. 1, pp 4 11.<br />

Kujansivu, P., and Lönnqvist, A. (2009) “Measuring the Impacts of an IC Development Service: The Case of the<br />

Pietari Business Campus”, Electr<strong>on</strong>ic Journal of Knowledge <strong>Management</strong>, Vol.7, No. 4, pp 469-480.<br />

Lönnqvist, A. and Pirttimäki, V. (2006) “The Measurement of Business Intelligence”, Informati<strong>on</strong> Systems<br />

<strong>Management</strong>, Vol. 23, No. 1, pp 32-40.<br />

Marin, J., and Poulter, A. (2004) “Disseminati<strong>on</strong> of Competitive Intelligence”, Journal of Informati<strong>on</strong> Science, Vol.<br />

30, No. 2, pp 165-180.<br />

McG<strong>on</strong>agle, J., and Vella, C. (2002) Bottom line Competitive Intelligence. Westport, CT: Quorum Books.<br />

Nicholas, R. (2004) “Knowledge <strong>Management</strong> Impacts <strong>on</strong> Decisi<strong>on</strong> Making Process”, Journal of Knowledge<br />

<strong>Management</strong>, Vol. 8, No. 1, pp 20-31.<br />

Nørreklit, H. (2000) “The Balance <strong>on</strong> the Balanced Scorecard: A Critical Analysis of Some of Its Assumpti<strong>on</strong>s”,<br />

<strong>Management</strong> Accounting Research, Vol. 11, 65-88.<br />

Pirttimäki, V., Lönnqvist, A., and Karjaluoto, A. (2006) “Measurement of Business Intelligence in Finnish<br />

Telecommunicati<strong>on</strong>s Company”, The Electr<strong>on</strong>ic Journal of Knowledge <strong>Management</strong>, Vol. 4, No. 1, pp 83-<br />

90.<br />

Poll, R, and Payne, P. (2006) “Impact Measures for Libraries and Informati<strong>on</strong> Services”, Library Hi Tech, Vol. 24,<br />

No. 4, pp 547-562.<br />

Prescott, J. E., and Bharwaj, G. (1995) “Competitive Intelligence Practices: A Survey”, Competitive Intelligence<br />

Review, Vol. 6, No. 2, pp 4-14.<br />

Qingjiu, T., and Prescott, J. E. (2000) “China: Competitive Intelligence Practices in an Emerging Market<br />

Envir<strong>on</strong>ment”, Competitive Intelligence Review, Vol. 11, No. 4, pp 65-78.<br />

Rothberg, H. and Ericks<strong>on</strong>, G. S. (2005) From Knowledge to Intelligence: Creating Competitive Advantage in the<br />

Next Ec<strong>on</strong>omy. Amsterdam: Elsevier Butterworth-Heinemann.<br />

Wright, S., and Calof, J. L. (2006) “The Quest for Competitive, Business and Marketing Intelligence: A Country<br />

Comparis<strong>on</strong> of Current Practices”, <str<strong>on</strong>g>European</str<strong>on</strong>g> Journal of Marketing, Vol. 40, No. 5-6, pp 453-465.<br />

Yin, R. K. (1994) Case Study Research: Designs and Methods (2nd ed.). Thousand Oaks, CA: Sage<br />

Publicati<strong>on</strong>s.<br />

390


The use of Virtual Public Space and eCommunities to Kick-<br />

Start eParticipati<strong>on</strong> – Timisoara, Romania<br />

M<strong>on</strong>ica Izvercianu and Ana-Maria Branea<br />

Politehnica University of Timisoara, Timisoara, Romania<br />

m<strong>on</strong>ica.izvercianu@mpt.upt.ro<br />

anabranea@yahoo.com<br />

Abstract: Romania passes through a period of administrative reorganizati<strong>on</strong> as it has to align itself to the<br />

regulati<strong>on</strong>s of the <str<strong>on</strong>g>European</str<strong>on</strong>g> Uni<strong>on</strong>. Top down decentralizati<strong>on</strong> and reorganizati<strong>on</strong> into regi<strong>on</strong>s and euro-regi<strong>on</strong>s<br />

must be coupled with a bottom up restructuring based <strong>on</strong> communities in order to near as much as possible the<br />

decisi<strong>on</strong> making process to the real problems. Even though for the <str<strong>on</strong>g>European</str<strong>on</strong>g> Uni<strong>on</strong>, as stated in the Bristol<br />

Accord, 2005, citizens’ participati<strong>on</strong> and sustainable communities are issues of major importance, in Romania<br />

these regulati<strong>on</strong>s are adopted but not appropriated. Citizens’ inquiries are thus carried <strong>on</strong> as to minimise the<br />

probability of appeals while “neighbourhood c<strong>on</strong>sulting councils” are political stepping st<strong>on</strong>es with no power in the<br />

administrative decisi<strong>on</strong> making process, unknown to the public and uninterested in c<strong>on</strong>sulting it. Citizens’<br />

participati<strong>on</strong> and the spirit of community are closely related to public space, but in Romania public space is dying,<br />

increasingly being used <strong>on</strong>ly for transfer and transformed into parking space. As during the 50 years under<br />

communist leadership an intense state policy was carried out to abolish communities and discourage any<br />

participati<strong>on</strong> unguided by the state, all forms of gatherings being prohibited for fear of uprising, public space was,<br />

and still is, viewed as bel<strong>on</strong>ging to the state and not the people. The younger generati<strong>on</strong>, untainted by the<br />

communist induced disregard for public space aband<strong>on</strong>s it instead of taking ownership, as it lives, an average of<br />

4 hours a day, in a virtual, global community hardly rooted in its physical locati<strong>on</strong>. In order to attain a community<br />

based restructuring of the administrati<strong>on</strong>, in this c<strong>on</strong>text, it is necessary to double the physical public space by a<br />

virtual <strong>on</strong>e and create a framework for public involvement. The use as incentive of a percentage of the city’s<br />

funds for citizen promoted projects to improve the quality of life in their communities, accessed through a<br />

competiti<strong>on</strong> <strong>on</strong> feasibility and public support is the <strong>on</strong>ly method of attracting interest and trust in the process.<br />

Keywords: community, eParticipati<strong>on</strong>, eGovernment, citizens' empowerment, virtual public space<br />

1. Introducti<strong>on</strong><br />

The process of reorganizing, decentralizing and alignment to the <str<strong>on</strong>g>European</str<strong>on</strong>g> Uni<strong>on</strong>’s regulati<strong>on</strong>s of new<br />

member states has been extensively studied in recent years.<br />

In Romania’s case new administrative levels, Regi<strong>on</strong>s and Euro regi<strong>on</strong>s, have been introduced.<br />

Despite their purpose of increasing efficiency, the lack of governing status or power per se has<br />

resulted in increased bureaucracy. At the same time, the <str<strong>on</strong>g>European</str<strong>on</strong>g> Uni<strong>on</strong>’s emphasis <strong>on</strong> citizens’<br />

participati<strong>on</strong> created tensi<strong>on</strong> and c<strong>on</strong>fusi<strong>on</strong>.<br />

Governments worldwide have increasingly become “closed bureaucratic instituti<strong>on</strong>s” having <strong>on</strong>ly<br />

sporadic c<strong>on</strong>tacts with their c<strong>on</strong>stituencies (Millard, 2009). They have, as suggested by Ferro and<br />

Molinari (Ferro & Molinari, 2010), developed a view <strong>on</strong> public services provisi<strong>on</strong> centred <strong>on</strong><br />

administrative fulfilment. C<strong>on</strong>sequently, a decrease in the public interest in political issues, democratic<br />

deficit (Sttefek, et al., 2008), was registered in direct relati<strong>on</strong> to the opaqueness of the decisi<strong>on</strong><br />

making process. Public participati<strong>on</strong> is regarded as a means of mending not <strong>on</strong>ly local problems<br />

through sustainable communities but also attempt to solve the widespread political deficit problem.<br />

Even though participati<strong>on</strong> might be vital to democracy, depending <strong>on</strong> each regi<strong>on</strong>’s historical<br />

background, it can be a sensitive or even c<strong>on</strong>troversial subject. In liberal cases self-interested political<br />

actors strive for private for private goals in a market like arena (Wiklung, 2005). However Romania<br />

still bears the marks of communitarian traditi<strong>on</strong>s where, under the pretext of supposed comm<strong>on</strong><br />

interests, an abuse of participati<strong>on</strong> occurred. Mass, homogenous, coercive and mandatory activities,<br />

supporting <strong>on</strong>ly the approved opini<strong>on</strong>s, left the populati<strong>on</strong> sceptical and reluctant to re-engage. The<br />

shift towards an individual heterogeneous participati<strong>on</strong>, in pursuit of both pers<strong>on</strong>al well being and<br />

grater comm<strong>on</strong> goods, is difficult and requires further research.<br />

The aim of this study is to present a framework for public participati<strong>on</strong> based <strong>on</strong> partnerships and<br />

coaliti<strong>on</strong>s, in a c<strong>on</strong>text of an increased blurring of the boundaries between and within public, private<br />

and n<strong>on</strong>-governmental/n<strong>on</strong>-profit sectors (Smith & Dalakiouridou, 2009) for an open, transparent and<br />

collaborative envir<strong>on</strong>ment for government-citizens-stakeholders interacti<strong>on</strong>s, defined as C<strong>on</strong>nected or<br />

391


M<strong>on</strong>ica Izvercianu and Ana-Maria Branea<br />

Networked Governance by the UN in 2008. The particular character of Romania’s history with<br />

participati<strong>on</strong> is taken into c<strong>on</strong>siderati<strong>on</strong> when investigating the citizen’s interest and willingness to<br />

participate.<br />

2. The <str<strong>on</strong>g>European</str<strong>on</strong>g> Uni<strong>on</strong>’s emphasis <strong>on</strong> participati<strong>on</strong> and Romania’s resp<strong>on</strong>se<br />

The importance of citizens’ participati<strong>on</strong> is internati<strong>on</strong>ally recognized both as direct individual<br />

participati<strong>on</strong> and as NGO mediated. The 2001 <str<strong>on</strong>g>European</str<strong>on</strong>g> Government: A White Paper (Commissi<strong>on</strong> of<br />

the <str<strong>on</strong>g>European</str<strong>on</strong>g> Communities, 2001) emphasises the need for a str<strong>on</strong>ger interacti<strong>on</strong> of regi<strong>on</strong>al, local<br />

governments and civil society as a resp<strong>on</strong>sibility of Member States. The same year The Citizens as<br />

Partners: OECD Guide to Informati<strong>on</strong>, C<strong>on</strong>sultati<strong>on</strong> and Public Participati<strong>on</strong> in Policy-Making,<br />

(Gramberger, 2001) defined three levels of citizen and authority cooperati<strong>on</strong>: Informing (unidirecti<strong>on</strong>al<br />

communicati<strong>on</strong> initiated by government or citizens), C<strong>on</strong>sultati<strong>on</strong> (acceptance by authorities of<br />

citizens’ feedback after previous informing) and Active participati<strong>on</strong> (implicati<strong>on</strong> of citizens in drafting<br />

public policy while the final decisi<strong>on</strong> remains with the authorities). The Code of Good Practice for Civil<br />

Participati<strong>on</strong> in the Decisi<strong>on</strong>-Making Process (Council of Europe, 2009), an analysis and identificati<strong>on</strong><br />

structure for the steps and actors involved in civil participati<strong>on</strong>, defines however four levels. While the<br />

first two are identical to those of the OECD guide the latter c<strong>on</strong>sist of Dialogue and Partnership, “the<br />

highest form of participati<strong>on</strong>”, (shared resp<strong>on</strong>sibility through co-decisi<strong>on</strong> making bodies).<br />

Based <strong>on</strong> the Arnstein’s ladder of citizens participati<strong>on</strong> (Arnstein, 2000) and a critical analysis of the<br />

above menti<strong>on</strong>ed classificati<strong>on</strong>s a participati<strong>on</strong> framework was developed, illustrating the proposed<br />

levels of involvement and the effect it can have <strong>on</strong> citizens. (fig.1)<br />

Through the eGovernment Acti<strong>on</strong> Plan 2011-2015 the <str<strong>on</strong>g>European</str<strong>on</strong>g> Commissi<strong>on</strong> promotes a new<br />

generati<strong>on</strong> of government services, open, flexible and collaborative, aimed at engaging and<br />

empowering <str<strong>on</strong>g>European</str<strong>on</strong>g> citizens and companies. The four priorities (Empower citizens and businesses,<br />

Reinforce mobility in the Single Market, Enable efficiency and effectiveness and Create the necessary<br />

key enablers and pre-c<strong>on</strong>diti<strong>on</strong>s to make things happen) are the basis for new internati<strong>on</strong>al,<br />

interoperable systems and key elements to form an inclusive and sustainable knowledge ec<strong>on</strong>omy as<br />

proposed in the Europe 2020 Strategy. It is also stipulated that by 2015 50% of citizens and 80% of<br />

companies should use eGovernment.<br />

Figure 1: Levels of participati<strong>on</strong> and effect <strong>on</strong> participants<br />

Unfortunately, Romania is not prepared, neither for participati<strong>on</strong> nor for eGovernment. Law 52/3003<br />

c<strong>on</strong>cerning public administrati<strong>on</strong>’s decisi<strong>on</strong> transparency, also known as the “sunshine law” dictates<br />

central and local government administrati<strong>on</strong>s to c<strong>on</strong>sult citizens and civil society organizati<strong>on</strong>s but<br />

under active participati<strong>on</strong> the minimum c<strong>on</strong>diti<strong>on</strong>s stated are open meetings and archived records of<br />

debates and nothing <strong>on</strong> actual citizen involvement. A 2011 government requirement to publish local<br />

budgets <strong>on</strong> the municipalities’ sites has uncovered that over 50% did not have a web site while<br />

approximately 80% of those which did <strong>on</strong>ly published informati<strong>on</strong> specifically required by law.<br />

392


M<strong>on</strong>ica Izvercianu and Ana-Maria Branea<br />

Figure 2: <str<strong>on</strong>g>European</str<strong>on</strong>g> eGovernment usage, source Eurostat<br />

2.1 Participati<strong>on</strong> and eParticipati<strong>on</strong> - opportunities, risks and challenges<br />

While collaborati<strong>on</strong> between government agencies was described by Bardoch in ’98 as an “unnatural<br />

act between two n<strong>on</strong> c<strong>on</strong>senting adults” the new level of openness, transparency and interacti<strong>on</strong><br />

needed brings the complexity of collaborati<strong>on</strong>s with both agencies and citizens to a whole new level.<br />

This is not c<strong>on</strong>diti<strong>on</strong>ed to unanimous participati<strong>on</strong> but to the existence of the opportunity to participate<br />

(Witteveen, 2000) which can be facilitated by supplementing traditi<strong>on</strong>al methods with ICT based <strong>on</strong>es<br />

removing the c<strong>on</strong>finement of time and space. However from the administrati<strong>on</strong>’s point of view it<br />

seems that cooperati<strong>on</strong> between citizens and the government in interactive policymaking is valuable<br />

as l<strong>on</strong>g as politicians can c<strong>on</strong>tinue to do their work and make the final decisi<strong>on</strong>s (Michels, 2006).<br />

Resistance is encountered from administrati<strong>on</strong>s and government agencies due to the loss of c<strong>on</strong>trol,<br />

labour intensity, difficulty in balancing access and security. Thus arises the need for a shift in the civil<br />

servants approach from a command and c<strong>on</strong>trol attitude to <strong>on</strong>e based <strong>on</strong> c<strong>on</strong>necti<strong>on</strong> and<br />

collaborati<strong>on</strong>, both internally and externally. (Friedman, 2007)<br />

Defined as “direct citizen involvement in, or influence over governmental processes” (Bucy &<br />

Gregs<strong>on</strong>, 2001) political participati<strong>on</strong> through the use of ICT, eParticipati<strong>on</strong>, enables citizens to take<br />

part in the decisi<strong>on</strong> making processes and develop social and political resp<strong>on</strong>sibility (Maier-Rabler &<br />

Huber, 2010) merging the top down (system oriented) and the bottom up (actor oriented)<br />

perspectives. Also referred to as <strong>on</strong>line public engagement, e-participati<strong>on</strong> can serve to encourage<br />

two-way communicati<strong>on</strong> between government and citizens, educate citizens about the rati<strong>on</strong>ale and<br />

complexity of policy-making, legitimize government decisi<strong>on</strong>s and provide opportunities for mutual<br />

learning (Coleman, S. and Gøtze, J., 2001).<br />

According to a comparative analysis <strong>on</strong> eParticipati<strong>on</strong> initiatives (Peart & Ramos-Diaz, 2008) the<br />

majority are characterized by a closer alignment to the governments interests than that of the citizens,<br />

a lack of guidance and learning support for novice users and more importantly poor opini<strong>on</strong><br />

aggregati<strong>on</strong> and visualizati<strong>on</strong> capabilities, namely traceability of <strong>on</strong>e’s c<strong>on</strong>tributi<strong>on</strong>s, other’s<br />

resp<strong>on</strong>ses <strong>on</strong> the same issue and the policy makers feedback and resp<strong>on</strong>ses in an easily navigable,<br />

multi-threaded, cross referenced database. This occurs due to the fact that generally, the <strong>on</strong>line<br />

service infrastructure is more of an electr<strong>on</strong>ic mirror of the physical <strong>on</strong>e, with no innovati<strong>on</strong> of internal<br />

working processes (automati<strong>on</strong>) based rather <strong>on</strong> available, <strong>on</strong> budget, technologies rather than the<br />

actual needs and expectati<strong>on</strong>s of the citizens. Coupled with a low promoti<strong>on</strong> of the infrastructure,<br />

limited to making it available, and resistance <strong>on</strong> the part of the administrative bodies, out of mistrust,<br />

lack of necessary skills and intensity of the work needed these characteristics result in high rates of<br />

perceived inefficiency and low citizen take up. (Ferro & Molinari, 2009) (Verdegem & Verleye, 2009)<br />

The issues of participati<strong>on</strong> and the knowledge gap between administrati<strong>on</strong>, experts and citizens,<br />

irrespective of educati<strong>on</strong> level, are similar, for eParticipati<strong>on</strong>, to those c<strong>on</strong>cerning the levels of “digital<br />

literacy” – the ability to access, navigate, critique and create c<strong>on</strong>tent through ICT (Mansell, et al.,<br />

2009) - not being equally distributed within society. The gap between the Digital Natives and Digital<br />

Emigrants (Prensky, 2001) based <strong>on</strong> differences in age, class, gender and educati<strong>on</strong> level can<br />

however be bridged through participati<strong>on</strong> in affinity spaces based <strong>on</strong> their skills and interests<br />

(Jenkins, et al., 2006) citizens related participati<strong>on</strong> being just a skills translati<strong>on</strong>. The <str<strong>on</strong>g>European</str<strong>on</strong>g><br />

Commissi<strong>on</strong> through its Digital Agenda (2010) identifies as a priority the fostering of digital literacy<br />

am<strong>on</strong>g citizens as a key competence in a knowledge based society. Another risk is the “low benefithigh<br />

costs scenario” (Curtin, 2007) namely the actual needs of each government for high levels of<br />

393


M<strong>on</strong>ica Izvercianu and Ana-Maria Branea<br />

participati<strong>on</strong> versus the populati<strong>on</strong>s demand and interest. Still other are related what Tsoukas defines<br />

as the “tyranny of light” obscuring citizens’ needs behind quantifiable indictors, targets and<br />

benchmarks. (Tsoukas, 1997)<br />

The benefits of participati<strong>on</strong> translate into increase in service efficiency and effectiveness through a<br />

better understanding of the public’s needs and desires, quality and legitimacy of the decisi<strong>on</strong> making<br />

process through greater transparency, awareness, acceptance and commitment to policies; active<br />

citizenship by generating social capital and mobilizing voluntary labour (Smith & Dalakiouridou, 2009);<br />

educati<strong>on</strong> of both citizens and politicians <strong>on</strong> their rights and resp<strong>on</strong>sibilities. To these eParticipati<strong>on</strong>,<br />

through the tools of social computing, usually referred to as Web 2.0 (Punie, 2009), adds a reducti<strong>on</strong><br />

in the costs to coordinate discussi<strong>on</strong>s and collaborati<strong>on</strong>s between the stakeholders, enhanced<br />

deliberativeness and informati<strong>on</strong> processing capabilities, exceeding all expectati<strong>on</strong>s when used as a<br />

tool for collaborative acti<strong>on</strong>s’ coordinati<strong>on</strong>. (Shirky, 2008) By facilitating co-creati<strong>on</strong> of public services<br />

eParticipati<strong>on</strong> enables the citizens’ transformati<strong>on</strong> from a c<strong>on</strong>sumer to a prosumer (Tapscott, 1995).<br />

2.2 Citizens’ participati<strong>on</strong> and sustainable communities<br />

The Bristol Accord, 2005, lists citizens’ participati<strong>on</strong> and sustainable communities as issues of major<br />

importance. These communities are defined as are places where people want to live and work, now<br />

and in the future as they meet the diverse needs of existing and future residents, are sensitive to their<br />

envir<strong>on</strong>ment and c<strong>on</strong>tribute to a high quality of life, are safe and inclusive, well planned, built and run<br />

and offer equality of opportunity and good services for all (The Office of the Deputy Prime Minister,<br />

2005). Am<strong>on</strong>g the eight established characteristics of a sustainable community, its method of<br />

governance “well run” ranks high, defined as having effective and inclusive participati<strong>on</strong>,<br />

representati<strong>on</strong> and leadership, in this order. Governance systems that must be representative and<br />

accountable are seen to equally facilitate leadership (strategic and visi<strong>on</strong>ary) and participati<strong>on</strong><br />

(inclusive, active, effective) of both individuals and organizati<strong>on</strong>s.<br />

The local level has become an excellent laboratory for democratic innovati<strong>on</strong>s (Al<strong>on</strong>so, 2009) as it is<br />

here that the state is most clearly seen as a “negotiating state” (March and Olsen, 1995) as there is a<br />

c<strong>on</strong>siderable ease of implementati<strong>on</strong> and lower risk in case of failure. According to Al<strong>on</strong>so<br />

governance at the local level requires the substituti<strong>on</strong> of the hierarchy, as an instrument of<br />

coordinati<strong>on</strong> with a variety of networks comprised of individual and collective actors with different<br />

degrees of instituti<strong>on</strong>alizati<strong>on</strong> (i.e. governance as an alternative to hierarchies) (Al<strong>on</strong>so, 2009)<br />

characterized by a c<strong>on</strong>tinuous c<strong>on</strong>sensus and group decisi<strong>on</strong>-making. The traditi<strong>on</strong>al hierarchies (top<br />

down governance) are substituted by Habermasian communicative rati<strong>on</strong>ality, which is grounded in<br />

negotiati<strong>on</strong> with and am<strong>on</strong>g resp<strong>on</strong>sible citizens. Traditi<strong>on</strong>ally local administrati<strong>on</strong>s make decisi<strong>on</strong>s in<br />

the name of the public interest, but often relying exclusively <strong>on</strong> “expert” knowledge for the structures<br />

of hierarchical coordinati<strong>on</strong> and administrative rati<strong>on</strong>ality proves to have little in comm<strong>on</strong> with the true<br />

or perceived necessities of the citizens and results in dissatisfacti<strong>on</strong> and distrust (Al<strong>on</strong>so, 2009).<br />

2.2.1 Participati<strong>on</strong> and communities in Timisoara<br />

Some forms of participati<strong>on</strong>, be it in pers<strong>on</strong> or through some use of ICT, are made available to the<br />

citizens of Timisoara but not facilitated and in some cases even hampered. Access to data is <strong>on</strong>e of<br />

the main complaints of would be participants. Even public enquires mandatory by law prior to<br />

approving any urban development plan have no guidelines for organizing, namely no minimum<br />

number of participants, feedback, metrics of a project’s exposure to the public or causes any<br />

questi<strong>on</strong>s to be raised if no objecti<strong>on</strong>s were made. Thus, these create the premises for enquiries for<br />

show, with public announcements of their due date in reader less newspapers and posters placed in<br />

areas that have no interest in the project so as to minimise the probability of appeals.<br />

Unfortunately, in Romania regulati<strong>on</strong>s are often adopted but not appropriated. In 2003,<br />

neighbourhood c<strong>on</strong>sulting councils were formed with the purpose of increasing the level of<br />

communicati<strong>on</strong>s between citizens and administrati<strong>on</strong> and that of citizens’ involvement in the decisi<strong>on</strong><br />

making process and neighbourhood event organizing, finding clear soluti<strong>on</strong>s to neighbourhood<br />

problems and encouraging social cohesi<strong>on</strong>. With no power in the decisi<strong>on</strong> making process these<br />

councils do no inspire citizens’ c<strong>on</strong>fidence and as it has happened in many cases worldwide were<br />

taken over by political parties. (Michels, 2006) becoming political stepping st<strong>on</strong>es, unknown to the<br />

public and uninterested in c<strong>on</strong>sulting it. The main problem laid in their basic structure oriented more<br />

towards a false representati<strong>on</strong>, <strong>on</strong>e is part of a council after filing a CV at the City Hall and being<br />

394


M<strong>on</strong>ica Izvercianu and Ana-Maria Branea<br />

approved, instead of a true citizens’ participati<strong>on</strong>. About half of these councils ceased their activity<br />

after a couple of years and most of the remaining <strong>on</strong>es have an average o three public<br />

announcements per year.<br />

2.2.2 Virtual public space<br />

As citizens’ participati<strong>on</strong> and the spirit of community are closely related to public space <strong>on</strong>e can easily<br />

assert the situati<strong>on</strong> by its state, as Romania public space is dying, increasingly being used <strong>on</strong>ly for<br />

transfer and transformed into parking space. More and more people move from home to car to work<br />

and back again hardly perceiving anything outside this system, therefore it is of no surprise if parking<br />

spaces gradually take over any available space (previous green areas, playgrounds, pedestrian<br />

pathways) and are a key criteri<strong>on</strong> in the search for a new residence even above apartment layout<br />

quality. As public space exists <strong>on</strong>ly c<strong>on</strong>diti<strong>on</strong>ed by the existence of private space, the past 50 years<br />

under communist rule and the following 20 of transiti<strong>on</strong> have left their mark <strong>on</strong> the Romanians’<br />

percepti<strong>on</strong> of it.<br />

If we c<strong>on</strong>sider the communism’s definiti<strong>on</strong> as a system of social organizati<strong>on</strong> based <strong>on</strong> the holding of<br />

all property in comm<strong>on</strong> with actual ownership ascribed to the community as a whole in reality it had a<br />

completely different effect <strong>on</strong> the percepti<strong>on</strong> of propriety as seen over the years following the<br />

Revoluti<strong>on</strong>. Having it forcefully taken away and then denied for many years left the Romanian<br />

populati<strong>on</strong> with a hunger for property that materialized in the highest percentage of privately owned<br />

homes in Europe 96%, followed by Lithuanians and Slovaks 89% and Hungarians with 87%, all<br />

having in comm<strong>on</strong> a similar former type of leadership.<br />

Public space was, and still is, viewed as bel<strong>on</strong>ging to the state and not the people. Despite the initial<br />

satisfactory facilities public space in residential units was provisi<strong>on</strong>ed with - playgrounds, squares,<br />

green areas - all were aband<strong>on</strong>ed when attenti<strong>on</strong> was focused <strong>on</strong> <strong>on</strong>e’s property, further widening the<br />

gap in the percepti<strong>on</strong> of public vs. private space.<br />

As during the 50 years under communist leadership an intense state policy was carried out to abolish<br />

communities and discourage any participati<strong>on</strong> unguided by the state, the result was a general state of<br />

distrust, a reminiscence of the fear of informants and no community spirit. Forced participati<strong>on</strong>,<br />

simulated enthusiasm for meetings and processi<strong>on</strong>s left a bitter taste and a general attitude of not<br />

voicing <strong>on</strong>es opini<strong>on</strong>s and disc<strong>on</strong>tents, half-heartedness and disbelief in better times and ir<strong>on</strong>ically an<br />

expectance of the state to fix every<strong>on</strong>e’s problems.<br />

The younger generati<strong>on</strong>, untainted by the communist induced disregard for public space aband<strong>on</strong>s it<br />

instead of taking ownership, as it lives, an average of 4 hours a day, in a virtual world hardly rooted in<br />

its physical locati<strong>on</strong>. Increasingly, the time spent <strong>on</strong>line surpasses in quantity that spent with the<br />

family and it affects all aspects of their life, social c<strong>on</strong>tacts and interacti<strong>on</strong>s. But regardless, the new<br />

generati<strong>on</strong> is willing and accustomed to being part of a community, participating and voicing <strong>on</strong>e’s<br />

opini<strong>on</strong>s and it could mean the salvati<strong>on</strong> of citizens’ participati<strong>on</strong> in Romania if a successful transfer of<br />

the virtual habits can be accomplished to a local network and after that to the real world.<br />

In order to attain a community based restructuring of the administrati<strong>on</strong>, in this c<strong>on</strong>text, it is necessary<br />

to create a framework for public involvement by doubling the physical public space by a virtual <strong>on</strong>e<br />

taking advantage of Romanians’, especially the younger generati<strong>on</strong>, interest in <strong>on</strong>line communities<br />

and collaborati<strong>on</strong>.<br />

Figure 3: Types of internet users (a) and evoluti<strong>on</strong> of social network users (b) in Romania<br />

395


M<strong>on</strong>ica Izvercianu and Ana-Maria Branea<br />

Even though according to Eurostat data almost 40% of Romanians do not have access to the internet<br />

through a computer, the “unplugged”, the allure of social networking sites decreases that number<br />

daily through the use of mobile devices. In little over 2 years, Facebook al<strong>on</strong>e reached 4,719,000<br />

users, over 21% of the entire Romanian populati<strong>on</strong> and 64,67% of its internet users. As the<br />

predominant users age are between 18 and 34 this provides an opportunity as it comprises the most<br />

active and pro-involvement groups. By classifying the types of internet users, creators/activists 13%,<br />

critics 19% joiners 24% and spectators 44%, the structure’s similarity to physical participatory acti<strong>on</strong>s<br />

and the potential it holds can be easily noticed. Affinity spaces, forming virtual communities, have<br />

been recognized as “opportunities for active citizenship” (Department of C<strong>on</strong>stituti<strong>on</strong>al Affairs, 2007)<br />

should the risk of disengaging people from “shared resp<strong>on</strong>sibilities, obligati<strong>on</strong>s and duties toward<br />

fellow citizens and the state” be overcome and address their shared interests in a way that stimulates<br />

them.<br />

To facilitate participati<strong>on</strong> and transform neighbourhoods into communities and thereafter sustainable<br />

<strong>on</strong>es it is necessary to translate <strong>on</strong>line communities to the local realm. The key to the <strong>on</strong>line<br />

counterparts lays in their structure based <strong>on</strong> comm<strong>on</strong> interests, passi<strong>on</strong> for photography, sports, or<br />

similarity in characteristics, young mothers, more defining than the physical locati<strong>on</strong>. Thus<br />

neighbourhood communities would be in fact a network of interlinked sub communities each<br />

bel<strong>on</strong>ging to o greater community from which it can derive support for comm<strong>on</strong> interests.<br />

In order to measure Timisoara’s citizens knowledge of the participati<strong>on</strong> means available and their<br />

willingness to use them a questi<strong>on</strong>naire research was carried out. Even though 96% of the<br />

resp<strong>on</strong>dents were interested in the problems facing their neighbourhood approximately 50% <strong>on</strong>ly<br />

knew of participati<strong>on</strong> and e-participati<strong>on</strong> tools available to them but could say little else and <strong>on</strong>ly 4%<br />

actually used them. 94% expressed a willingness to participate, of which most aged between 18 and<br />

45 favoured ICT means while resp<strong>on</strong>dents over 45 generally preferred traditi<strong>on</strong>al methods, physical<br />

presence at the debate.<br />

Based <strong>on</strong> a cross analysis of the participants interest in their neighbourhoods’ problems and their<br />

willingness to participate in the decisi<strong>on</strong> making process it can be noted that the majority of the<br />

“somewhat interested” and over half of the “not interested” would participate through ICT tools should<br />

they be encouraged. Divided by age groups, the most willing to participate are fortunately the most<br />

abundant internet users, aged 18-34. However when asked about their knowledge of the means<br />

available to them and to which they would turn in case of a problem the preferred choice was the<br />

neighbourhood councils (NC), proving the feasibility of the decentralizati<strong>on</strong> c<strong>on</strong>cept. Unfortunately the<br />

vast majority of the <strong>on</strong>es turning to NCs were those who either knew of their existence but little else or<br />

who had never heard of them previously, a proof of their malfuncti<strong>on</strong> and c<strong>on</strong>sequent citizens’<br />

mistrust.<br />

Figure 4: Preference for traditi<strong>on</strong>al or ICT enabled participati<strong>on</strong> based <strong>on</strong> interest in neighbourhood<br />

problems (a) and age group (b)<br />

The general c<strong>on</strong>clusi<strong>on</strong> of the questi<strong>on</strong>naire research was a high interest and willingness to<br />

participate mostly through the use of ICT but low initiative and an even lower knowledge of their rights<br />

and opti<strong>on</strong>s. In order to achieve an effective civic participati<strong>on</strong> a three stage process is needed, each<br />

stage deriving from the completi<strong>on</strong> of the previous <strong>on</strong>e: informati<strong>on</strong> – citizens’ educati<strong>on</strong> <strong>on</strong> the local<br />

government’s policies, strategies, projects and the means to participate; c<strong>on</strong>trol – accountability of<br />

396


M<strong>on</strong>ica Izvercianu and Ana-Maria Branea<br />

local government resulted from increased transparency of acti<strong>on</strong>s and public interest and<br />

involvement; participati<strong>on</strong> – participative and deliberative democracy through citizens’ empowerment.<br />

Figure 5: Preference for partners in solving neighbourhood problems based <strong>on</strong> knowledge of<br />

neighbourhood council activity<br />

The key, however, in attracting interest and earning the citizens’ trust in the process and it can be<br />

achieved through the use as incentive of a percentage of the city’s funds for citizen promoted projects<br />

to improve the quality of life in their communities, accessed through a competiti<strong>on</strong> <strong>on</strong> feasibility and<br />

public support.<br />

Based <strong>on</strong> a comparis<strong>on</strong> of three well known participatory budgeting initiatives, which according to<br />

Sintomer are a paradigm for participati<strong>on</strong>, (Sintomer, et al., 2008), especially at the local level,<br />

allowing citizens to have a say in the way the city budget is spent, we devised the framework of an<br />

eParticipati<strong>on</strong> process based <strong>on</strong> Romania’s needs and characteristics. In the case of Brazil’s Porto<br />

Alegre, the citizens are able to state their preferences for the city’s future projects, forming thematic<br />

investment categories, and vote <strong>on</strong> their individual regi<strong>on</strong>’s priorities, creating the basics of a<br />

budgeting matrix. This in turn divides the available funds of each thematic category to the 17 city<br />

regi<strong>on</strong>s based <strong>on</strong> their populati<strong>on</strong>, dysfuncti<strong>on</strong>s and selected priorities. In Solford, UK, the final<br />

decisi<strong>on</strong> over the city’s budget is taken by the City Council using a resource matrix based <strong>on</strong> the<br />

needs expressed by the citizens, by post or <strong>on</strong>line, and the areas’ needs. In the third case, Getafo,<br />

Spain, citizens are able to both create proposals and vote for their favourite. The top five are<br />

afterwards examined by the local authorities’ technical staff to determine their technical, ec<strong>on</strong>omical<br />

and legal feasibility and through a public debate are approved for implementati<strong>on</strong>. (Alfaro, et al.,<br />

2010)<br />

In Timisoara’s case the citizens are not prepared for the resp<strong>on</strong>sibilities endowed by the Brazilian<br />

model while the other two are limited, the Salford model using citizens as a database for perceived<br />

needs and problems while the Spanish <strong>on</strong>e limits their power to a restricted number of projects.<br />

A compilati<strong>on</strong> of these initiatives represents the base of our process with a particularity of limiting the<br />

level of participati<strong>on</strong> based <strong>on</strong> a project’s scale, to ensure that the correct know how and expert<br />

knowledge is applied and limit potential backfires. Therefore, for priority <strong>on</strong>e city projects, such as<br />

large infrastructure projects, the citizens’ participati<strong>on</strong> is limited to drafting the city’s hierarchy of<br />

priorities and taking part in public debates, while the final decisi<strong>on</strong>, arbitrati<strong>on</strong>, is left in the hands of<br />

the local authorities. However for smaller scale, community level problems, the soluti<strong>on</strong> can be mainly<br />

citizen generated, validated by the local authorities and voted in a city wide competiti<strong>on</strong> for funds,<br />

backed by a negotiati<strong>on</strong> stage security fail safe.<br />

397


M<strong>on</strong>ica Izvercianu and Ana-Maria Branea<br />

Figure 6: Proposed two stage framework for participati<strong>on</strong><br />

3. C<strong>on</strong>clusi<strong>on</strong><br />

Through participati<strong>on</strong> and more feasibly e-participati<strong>on</strong>, government accountability and project<br />

subsidiarity can be accomplished together with a balance between ec<strong>on</strong>omic competitiveness, social<br />

cohesi<strong>on</strong> and envir<strong>on</strong>mental quality. In order to overcome the Romania’s participati<strong>on</strong> deficiency it is<br />

necessary to adapt the strategy for citizen empowerment, as required by the <str<strong>on</strong>g>European</str<strong>on</strong>g> Uni<strong>on</strong>, to the<br />

younger generati<strong>on</strong>, its problems but most importantly the communicati<strong>on</strong> means it already uses.<br />

Scepticism and reluctance to engage, a result of Romania’s experience with participati<strong>on</strong> and the<br />

opaqueness of the current decisi<strong>on</strong> making process, can <strong>on</strong>ly be dissipated through immediately<br />

visible results to citizens’ proposals, making local, small scale interventi<strong>on</strong>s the most efficient.<br />

Acknowledgements<br />

“This work was partially supported by the strategic grant POSDRU 107/1.5/S/77265, inside<br />

POSDRU Romania 2007-2013 co-financed by the <str<strong>on</strong>g>European</str<strong>on</strong>g> Social Fund – Investing in People.”<br />

References<br />

Alfaro, C., Gomez, J., Lavin, J. M. & Molero, J. J., 2010. A c<strong>on</strong>figurable architecture for e-participatory budgeting<br />

support. eJournal of eDemocracy, 2(1), pp. 39-45.<br />

Al<strong>on</strong>so Á. I. (2009) e-Participati<strong>on</strong> and local governance: case study, Theoretical and Empirical Researches in<br />

Urban <strong>Management</strong>, Number 3(12) / August 2009, CCASP TERUM, Pp. 49-62.<br />

Arnstein, S. R., (2000). A Latter of Citizen Participati<strong>on</strong>. In: R. T. Gates & F. Stout, eds. The City Reader: 2nd<br />

Editi<strong>on</strong>. New York: Routledge, pp. 240-252.<br />

398


M<strong>on</strong>ica Izvercianu and Ana-Maria Branea<br />

Bristol Accord, (2005) C<strong>on</strong>clusi<strong>on</strong>s of Ministerial Informal <strong>on</strong> Sustainable Communities in Europe, UK<br />

PRESIDENCY, The Office of the Deputy Prime Minister<br />

Bucy, E. & Gregs<strong>on</strong>, K., (2001). Media Participati<strong>on</strong>: A Legitimizing Mechanism of Mass Democracy. New Media<br />

& Society, 3(3), pp. 357-380.<br />

Commissi<strong>on</strong> of the <str<strong>on</strong>g>European</str<strong>on</strong>g> Communities, (2001). <str<strong>on</strong>g>European</str<strong>on</strong>g> Government: A White Paper, Brussels<br />

Council of Europe, (2009). Code of Good Practice for Civil Participati<strong>on</strong> in the Decisi<strong>on</strong>-making Process,<br />

<str<strong>on</strong>g>C<strong>on</strong>ference</str<strong>on</strong>g> of INGOs of the Council of Europe.<br />

Coleman, S. and Gøtze, J., (2001), Bowling Together: Online Public Engagement in Policy Deliberati<strong>on</strong>, Hansard<br />

Society, L<strong>on</strong>d<strong>on</strong>, UK (2001); http://bowlingtogether.net/bowlingtogether.pdf.]<br />

Curtin, D., (2007). Transparency, audiences and the evolving role of the EU Council of Ministers. In: J. Fossum &<br />

P. Schlesinger, ed. The <str<strong>on</strong>g>European</str<strong>on</strong>g> Uni<strong>on</strong> and the public sphere: a communicative space in the making?.<br />

L<strong>on</strong>d<strong>on</strong>: Routledge, pp. 246-258.<br />

Department of C<strong>on</strong>stituti<strong>on</strong>al Affairs, (2007). The Future of Citizenship, United Kingdom<br />

Eder, K., (2007). The public sphere and <str<strong>on</strong>g>European</str<strong>on</strong>g> democracy. Mechanisms of democratisati<strong>on</strong> in the<br />

transnati<strong>on</strong>al situati<strong>on</strong>. in: J. Fossum & P. Schlesinger, ed. The <str<strong>on</strong>g>European</str<strong>on</strong>g> Uni<strong>on</strong> and the public sphere: a<br />

communicative space in the making?. L<strong>on</strong>d<strong>on</strong>: Routledge, pp. 44-64<br />

Ferro, E. & Molinari, F., (2009). Making Sense of Gov 2.0 Strategies: ’No Citizens, No Party’. Vienna,<br />

Ferro, E. & Molinari, F., (2010). Framing Web 2.0 in the Process of Public Sector Innovati<strong>on</strong>: Going Down the<br />

Participati<strong>on</strong> Ladder. <str<strong>on</strong>g>European</str<strong>on</strong>g> Journal of ePractice • www.epracticejournal.eu, March.Volume 9.<br />

Friedman, T., (2007). The world is flat 3.0: A brief history of the twenty-first century. New York: Picador.<br />

Gramberger, M., (2001). Citizens as Partners OECD Handbook <strong>on</strong> Informati<strong>on</strong>, C<strong>on</strong>sultati<strong>on</strong> and Public<br />

Participati<strong>on</strong> in Policy-making, Paris: Organisati<strong>on</strong> for ec<strong>on</strong>omic co-operati<strong>on</strong> and development.<br />

Jenkins, H.; Clint<strong>on</strong>, K.; Purushotma, R.; Robis<strong>on</strong>, A.; Weigel, M., (2006). C<strong>on</strong>fr<strong>on</strong>ting the Challenges of<br />

Participatory Culture : Media Educati<strong>on</strong> for the 21st Century.<br />

Liebert, U., (2007). Transnati<strong>on</strong>alising the public sphere? The <str<strong>on</strong>g>European</str<strong>on</strong>g> Parliament, promises and anticipati<strong>on</strong>s.<br />

In: J. Fossum & P. Schlesinger, ed. The <str<strong>on</strong>g>European</str<strong>on</strong>g> Uni<strong>on</strong> and the public sphere: a communicative space in<br />

the making?. L<strong>on</strong>d<strong>on</strong>: Routledge, pp. 259-278.<br />

March, J. and Olsen, J. (1995), Democratic Governance, Free Press, New York.<br />

Mansell, R., Avgerou, C., Quah, D. & Silverst<strong>on</strong>e, R., (2009). The Oxford Handbook of Informati<strong>on</strong> and<br />

Communicati<strong>on</strong> Technologies. Oxford: Oxford University Press.<br />

Michels, Ank M. B., (2006) Citizen participati<strong>on</strong> and democracy in the Netherlands, Democratizati<strong>on</strong>, Vol.13,<br />

No.2, April 2006, pp.323–339, ISSN 1351-0347, Taylor & Francis<br />

Millard, J., (2009). Everyday Government. Malmo, Sweden,<br />

Peart, M. & Ramos-Diaz, J., (2008). Taking Stock: Local e-democracy in Europe and the USA. Internati<strong>on</strong>al<br />

Journal of Electr<strong>on</strong>ic Governance, 1(4), pp. 400-433.<br />

Prensky, M., (2001). Digital Natives, Digital Immigrants. On the Horiz<strong>on</strong>, 9(5).<br />

Punie, Y.; Misuraca, G.; Osimo, D, (2009). Public Services 2.0: The Impact of Social Computing <strong>on</strong> Public<br />

Services, s.l.: JRC Scientific and Technical Reports. <str<strong>on</strong>g>European</str<strong>on</strong>g> Commissi<strong>on</strong>, Joint Research Centre,<br />

Institute for Prospective Technological Studies.<br />

Shirky, C., (2008). Here comes everybody: The power of organizing without organizati<strong>on</strong>s. Penguin Press Group.<br />

Sintomer, Y., Herzberg, C. & Röcke, A., 2008. Participatory budgeting in Europe: potentials and challenges.<br />

Internati<strong>on</strong>al Journal of Urban Regi<strong>on</strong>al Reearch, Volume 32, pp. 164-178.<br />

Smith, S. & Dalakiouridou, E., (2009). C<strong>on</strong>textualising Public (e)Participati<strong>on</strong> in the Governance of the <str<strong>on</strong>g>European</str<strong>on</strong>g><br />

Uni<strong>on</strong>. <str<strong>on</strong>g>European</str<strong>on</strong>g> Journal of ePractice, Volume 7. March.<br />

Sttefek, J., Kissling, C. & Nanz, P., (2008). Civil Society Participati<strong>on</strong> in <str<strong>on</strong>g>European</str<strong>on</strong>g> and Global Governance: A<br />

Cure for the Democratic Deficit?. Basingstoke: Palgrave Macmillan.<br />

Tapscott, D., (1995). The Digital Ec<strong>on</strong>omy in the New Network Ec<strong>on</strong>omy: Promise and Peril in the Age of<br />

Networked Intelligence. New York: Mcgraw-Hill.<br />

Tsoukas, H., (1997). The tyranny of light. The temptati<strong>on</strong>s and the paradoxes of the informati<strong>on</strong> society. Futures,<br />

29(9), pp. 827-843.<br />

Verdegem, P. & Verleye, G., (2009). User-centred E-Government in practice: A comprehensive model for<br />

measuring user satisfacti<strong>on</strong>. Government Informati<strong>on</strong> Quarterly, Volume 26, p. 487–497.<br />

Wiklung, H., (2005). A Habermasian analysis of the deliberative-democratic potential of ICT-enabled services in<br />

Swedish municipalities. New media & society, 7(2), pp. 247-270.<br />

399


Strategic <strong>Management</strong> and Informati<strong>on</strong> Evaluati<strong>on</strong><br />

Challenges Facing Entrepreneurs of SMEs in ICT<br />

Maroun Jneid 1 and Antoine Tannous 2<br />

1 Doctoral school Cogniti<strong>on</strong>-Language-Interacti<strong>on</strong>, Université Paris 8, Saint<br />

Denis, France<br />

Faculty of engineering, Ant<strong>on</strong>ine University, Beirut, Leban<strong>on</strong><br />

2 Marketing Department, faculty of Business, Lebanese University, Tripoli,<br />

Leban<strong>on</strong><br />

mjneid@etud.univ-paris8.fr<br />

antoine_tannous@yahoo.fr<br />

Abstract: Achievements of young Entrepreneurs have impact <strong>on</strong> a country development, thus the governments<br />

are creating awareness about entrepreneurship to encourage young people to choose Entrepreneurship as a<br />

career path therefore number of entrepreneurs is growing c<strong>on</strong>tinuously; while a start-up company exists in a<br />

world of uncertainties and needs two to four years to break even then another two years to become stable so the<br />

survival of the entrepreneurs in the early stage rely more <strong>on</strong> an efficient strategic management, competitive<br />

advantages and well organized internal knowledge management. The entrepreneurial process model for small<br />

and medium business in its related stages is based <strong>on</strong> activities, the efficiency of a stage-activity depends <strong>on</strong> the<br />

accuracy of the informati<strong>on</strong> evaluati<strong>on</strong> and differs from an ec<strong>on</strong>omy to another; a pertinent evaluati<strong>on</strong> of the<br />

external envir<strong>on</strong>ment influences the efficiency of strategic management in a start-up company. This paper<br />

presents the challenges in informati<strong>on</strong> evaluati<strong>on</strong> and strategic management facing entrepreneurship since the<br />

early-stage, the important role of the Informati<strong>on</strong> System (IS) in the survival of the entrepreneur business; In<br />

additi<strong>on</strong>, to the ec<strong>on</strong>omic c<strong>on</strong>text elements to be c<strong>on</strong>sidered when designing a Competitive Intelligence system<br />

soluti<strong>on</strong>. This study is based <strong>on</strong> a qualitative approach and will be of use for the entrepreneurs of SMEs that<br />

needs to stand <strong>on</strong> a competitive edge since their early stage but they have lack of awareness and limited<br />

capabilities to invest in the integrati<strong>on</strong> of a Competitive Intelligence and Informati<strong>on</strong> Evaluati<strong>on</strong> process systems.<br />

Keywords: competitive intelligence system, efficiency-driven ec<strong>on</strong>omy, entrepreneurial process, informati<strong>on</strong><br />

evaluati<strong>on</strong>, strategic management<br />

1. Introducti<strong>on</strong><br />

In this globalizati<strong>on</strong> era, a start-up company exists in a world of uncertainties and the management of<br />

the survival phase is becoming more complex and rely more <strong>on</strong> efficient strategic management and<br />

competitive advantage.<br />

We will present the informati<strong>on</strong> evaluati<strong>on</strong> and strategic management challenges facing<br />

entrepreneurship since the early-stage, the role of the Informati<strong>on</strong> System (IS) and competitive<br />

advantage to gain; In additi<strong>on</strong>, to the ec<strong>on</strong>omic c<strong>on</strong>text elements to be c<strong>on</strong>sidered when designing a<br />

Competitive Intelligence system soluti<strong>on</strong>.<br />

This article is divided into three secti<strong>on</strong>s; in the first secti<strong>on</strong> the entrepreneurial process and<br />

challenges c<strong>on</strong>text are presented, in the sec<strong>on</strong>d secti<strong>on</strong> the challenges and opportunities in an<br />

efficiency-driven ec<strong>on</strong>omy are presented and in the third secti<strong>on</strong> the result of a qualitative research<br />

c<strong>on</strong>ducted <strong>on</strong> 24 entrepreneurs in their early-stage is presented.<br />

2. Entrepreneurial of SME process<br />

Based <strong>on</strong> Moore’s model, Bygrave (2004) presents the entrepreneurial process as a set of stages and<br />

events that follow <strong>on</strong>e another. These stages are: the idea or c<strong>on</strong>cepti<strong>on</strong> of the business, the event<br />

that triggers the operati<strong>on</strong>s, implementati<strong>on</strong> and growth. In his “Model of the entrepreneurial process”<br />

Bygrave (2004) presented a framework that highlights the critical factors that drive the development of<br />

the business at each phase, and according to Bygrave (2004, p5), entrepreneurial traits are shaped<br />

by pers<strong>on</strong>al attributes and envir<strong>on</strong>ment factors like opportunities, competiti<strong>on</strong>, resources, customers,<br />

etc.<br />

The figure below presents the entrepreneurial process with the related attributes and factors.<br />

400


Maroun Jneid and Antoine Tannous<br />

Source: Bygrave, W.D., (2009). The Entrepreneurial Process. In The Portable MBA in<br />

Entrepreneurship. Hoboken, NJ, USA: John Wiley & S<strong>on</strong>s.<br />

Figure 1: Model of the entrepreneurial process<br />

3. Strategic management and informati<strong>on</strong> evaluati<strong>on</strong> challenges facing the<br />

entrepreneurs<br />

At each of the above stages in the Entrepreneurial process, there are serious informati<strong>on</strong> evaluati<strong>on</strong><br />

challenges facing the entrepreneurs, Said Hussein & Maryse Salles (2003) have grouped the<br />

informati<strong>on</strong> requirements into the thirteen topics: 1.Market opportunities 2.Anticipati<strong>on</strong> of the market<br />

behaviour 3.C<strong>on</strong>tinuous ameliorati<strong>on</strong> of skills <strong>on</strong> the acquired markets 4.Identificati<strong>on</strong> of the new<br />

qualified human resources 5.Partnership possibility with other companies 6.Partnership possibility<br />

7.Foreign market opportunities 8.Ameliorati<strong>on</strong> of skills regarding the external market 9.Adaptati<strong>on</strong><br />

c<strong>on</strong>straints <strong>on</strong> the exportati<strong>on</strong> markets regarding competitors 10.identificati<strong>on</strong> of competitors<br />

11.Competitor functi<strong>on</strong>al strength and weakness 12.Adaptati<strong>on</strong> capacity regarding envir<strong>on</strong>mental<br />

change 13.Latest informati<strong>on</strong> and communicati<strong>on</strong> technologies. This informati<strong>on</strong> will be the source for<br />

supporting any strategic decisi<strong>on</strong> and a Competitive Intelligence (CI) system tool. According to<br />

Grabova (2010), a CI system is the organizati<strong>on</strong>al process for systematically collecting, processing,<br />

analysing, and distributing informati<strong>on</strong> to decisi<strong>on</strong> makers about an organizati<strong>on</strong>’s external<br />

envir<strong>on</strong>ment. And, according to Hohhof (1994) a CI system may track competitors, markets,<br />

technological developments and sources.<br />

According to Hohhof (1994), Informati<strong>on</strong> is analysed in a specific envir<strong>on</strong>ment of problems,<br />

transformed into “intelligence,” and delivered to decisi<strong>on</strong> makers. In order to be c<strong>on</strong>sidered<br />

intelligence, informati<strong>on</strong> must be relevant to the decisi<strong>on</strong> at hand and must support an acti<strong>on</strong> or<br />

decisi<strong>on</strong>. Unfocused or “nice-to-know” informati<strong>on</strong> is not appropriate in CI activities.<br />

However, SITE (2007) c<strong>on</strong>siders the following challenges: how and what informati<strong>on</strong> would be<br />

collected? How the informati<strong>on</strong> would appear when presented? Where it would be distributed? And<br />

who would have access to it?<br />

Nowadays, Competitive Intelligence becomes an essential part of any enterprise, even an SME or an<br />

entrepreneurial in its early-stage. According to Grabova (2010) this necessity is caused by the<br />

increasing data volume indispensable for decisi<strong>on</strong> making. Existing soluti<strong>on</strong>s and tools are aimed at<br />

large-scaled enterprises; thereby they are inaccessible or insufficient for SMEs because of high price,<br />

401


Maroun Jneid and Antoine Tannous<br />

complexity, and high infrastructure requirements. SMEs require light and affordable CI system<br />

soluti<strong>on</strong>.<br />

4. Efficiency-driven ec<strong>on</strong>omy factors and impact <strong>on</strong> the entrepreneurs in their<br />

early-stage, the case of Leban<strong>on</strong><br />

According to Schwab (2011), Leban<strong>on</strong>’s ec<strong>on</strong>omy is classified as an ec<strong>on</strong>omy in transiti<strong>on</strong> from<br />

efficiency-driven stage of development to innovati<strong>on</strong>-driven stage of development in three stages<br />

classificati<strong>on</strong> of ec<strong>on</strong>omic development; and according to Dutta (2011) Leban<strong>on</strong> is included for the<br />

first time in the ICT readiness report and held the rank 95 in the ICT readiness index over 138<br />

ec<strong>on</strong>omies studied in the 2011 report.<br />

As stated by BMI (2011) a c<strong>on</strong>tinuous growth of 12% in ICT Lebanese market is forecasted till 2015.<br />

And “am<strong>on</strong>g the seven countries studied from the MENA regi<strong>on</strong>, Lebanese adults are the most likely<br />

to foresee good opportunities for starting a business, and perceive they have the knowledge, skills<br />

and experience to do so” stated (IDRC, 2010).<br />

According to Porter (1980) entrepreneurs are more attracted by the competitive forces presented in<br />

certain ec<strong>on</strong>omy sectors such as industry growth, c<strong>on</strong>centrati<strong>on</strong>, and rivalry and entry barriers. The<br />

c<strong>on</strong>tinuous ec<strong>on</strong>omic growth of ICT sector in the global ec<strong>on</strong>omy attracts the entrepreneurs and there<br />

are potential opportunities for entrepreneurship in ICT sector such as telecommunicati<strong>on</strong>s, banking,<br />

utilities, real state and government, but it depends <strong>on</strong> the political stability needed to implement<br />

ec<strong>on</strong>omic reforms in the country from <strong>on</strong>e side and <strong>on</strong> the capabilities of the Entrepreneur to survive<br />

the challenges in informati<strong>on</strong> evaluati<strong>on</strong> and strategic management in the early-stage and <strong>on</strong><br />

recognising the important role of the Informati<strong>on</strong> System (IS) and the competitive advantage which is<br />

the issue of our study.<br />

5. Qualitative research<br />

One aspect of this qualitative study is to explore the role of the informati<strong>on</strong> technology in facing the<br />

informati<strong>on</strong> evaluati<strong>on</strong> challenges and supporting an efficient strategic management. A qualitative<br />

approach was selected for this study because if suits better our field of study. The targeted actors was<br />

60 entrepreneurs in their early-stage qualified to the sec<strong>on</strong>d round of a competiti<strong>on</strong> of the best<br />

entrepreneur business plan in north regi<strong>on</strong> of Leban<strong>on</strong>, candidates undergo a rigorous selecti<strong>on</strong><br />

process and must meet specific criteria prior to being selected to the sec<strong>on</strong>d round of a three rounds<br />

competiti<strong>on</strong>. 24 of 60 qualified entrepreneurs have answered our questi<strong>on</strong>naire half of them were<br />

entrepreneur in the ICT sector.<br />

6. Results<br />

We all know that decisi<strong>on</strong>s maker needs informati<strong>on</strong> to support their decisi<strong>on</strong>s making, but a critical<br />

element of this study involved exploring the strategic view of an entrepreneur in the early-stage, his<br />

awareness about collecting informati<strong>on</strong> about the competitors’ c<strong>on</strong>text and about the supporting tools<br />

and his capability to achieve these complicated tasks. So all the questi<strong>on</strong>s were asked in the same<br />

logic, beginning with the current Available Informati<strong>on</strong> (A.I.) the entrepreneur has about the specific<br />

topic than followed by the Needed Informati<strong>on</strong> (N.I.) that the entrepreneur is searching for in the<br />

related topic than the Informati<strong>on</strong> Collecti<strong>on</strong> Methods and Tools (I.C.M.T.) that the entrepreneur is<br />

intending to use.<br />

The table below recapitulates the informati<strong>on</strong> evaluati<strong>on</strong> of the related topics as described by the<br />

interrogated entrepreneurs.<br />

Table 1: Questi<strong>on</strong>naire recapitulati<strong>on</strong><br />

Informati<strong>on</strong> Evaluati<strong>on</strong> (available, needed, collecti<strong>on</strong> methods and tools) and Analysis<br />

1. market opportunities<br />

(A.I.): A minority of the entrepreneurs have not identified their target markets. In other hand half the<br />

entrepreneurs, their target markets are not adapted to their ideas. Only 9 entrepreneurs specify their<br />

market niches identificati<strong>on</strong> and 11 the market to avoid. 13 of them d<strong>on</strong>’t explain based <strong>on</strong> what they<br />

choose their market. Knowing that, 21 of them have not d<strong>on</strong>e market research to select their target<br />

markets.<br />

C<strong>on</strong>sequently, we deduce the existence of lack of objectivity of entrepreneurs, in their choice of target<br />

markets. They were not based <strong>on</strong> market research to verify the choice of their target market<br />

(N.I.): The majority of the entrepreneurs seek informati<strong>on</strong> about the market size, market growth, market<br />

402


Maroun Jneid and Antoine Tannous<br />

needs and life cycle, c<strong>on</strong>sumers’ revenue, the social classes and their purchase power, how to calculate<br />

the risk, the Lebanese governmental rules and regulati<strong>on</strong>s, etc. they also search for specific informati<strong>on</strong><br />

related to their field of activities such as:<br />

“Recycling factories and storages; The business model of interior engineers’ c<strong>on</strong>tractors; The natural gas<br />

market in the regi<strong>on</strong>; Etc.”<br />

(I.C.M.T.): The majority of the answers were as follow: Survey, interviewing experienced people, focus<br />

group, qualitative and quantitative customers’ studies, search informati<strong>on</strong> form associati<strong>on</strong>s, market<br />

opportunity analysis, and sec<strong>on</strong>dary data about existing surveys published <strong>on</strong>line, attending all events and<br />

forums, to ph<strong>on</strong>e c<strong>on</strong>tacts’ suppliers and intermediaries, statistics, c<strong>on</strong>sultant, market research companies,<br />

observati<strong>on</strong>al research method<br />

2. anticipati<strong>on</strong> of the market behaviour<br />

(A.I.): The majority of the entrepreneurs aren’t able to identify their actual needs of informati<strong>on</strong>. Their<br />

replies have nothing to do with the questi<strong>on</strong>. The same situati<strong>on</strong> appears for the sec<strong>on</strong>d part of the<br />

questi<strong>on</strong>. They aren’t able to specify their need of informati<strong>on</strong> regarding the anticipati<strong>on</strong> of the market<br />

behaviour. This means that they have no awareness of the importance of an informati<strong>on</strong> system in the<br />

success of a company<br />

(N.I): Few entrepreneurs d<strong>on</strong>’t give answer. The majority of them specify many needs in informati<strong>on</strong> related<br />

to qualitative and quantitative market research as follow: “Weekly and m<strong>on</strong>thly ec<strong>on</strong>omic Statistics and<br />

surveys. More informati<strong>on</strong> <strong>on</strong> Lebanese users <strong>on</strong>line. Industry behavioural report through the last 5 years.<br />

How much the product is close to client need? The size of the market. The size of potential new entries;<br />

data c<strong>on</strong>cerning the introducti<strong>on</strong> of a new product <strong>on</strong> the market; Etc.”<br />

(I.C.M.T.): Answers to this questi<strong>on</strong> are quite similar to the collecti<strong>on</strong> method and tools for market<br />

opportunities, moreover some entrepreneurs added the following: “M<strong>on</strong>thly ec<strong>on</strong>omic reports, direct c<strong>on</strong>tact<br />

with my customers, existing reading and research (sec<strong>on</strong>dary data) as well <strong>on</strong>line questi<strong>on</strong>naire and data,<br />

follow up by direct call, interview with specialist.; To c<strong>on</strong>tact universities since the majority of the startups<br />

comes from there; Local marketing company; Etc.”<br />

3. c<strong>on</strong>tinuous ameliorati<strong>on</strong> of skills <strong>on</strong> the acquired markets<br />

(A.I.):<br />

i. How you are commercializing your products<br />

The majority of the entrepreneurs d<strong>on</strong>’t have accurate informati<strong>on</strong> to look for during the marketing of their<br />

products. This mean they have weaknesses to identifies needed informati<strong>on</strong> in this phase. They ignore<br />

what type of informati<strong>on</strong> they need for their successes.<br />

ii. The latest technologies in your business domain<br />

Few of entrepreneurs indicate their needs in the latest technologies in their business domain as follow:<br />

“Ph<strong>on</strong>e applicati<strong>on</strong>; A marketplace system has been implemented by WePay. We are in direct c<strong>on</strong>tact with<br />

them; the new wireless mixing technology of perfumes; Website <strong>on</strong>line, inside store PC and later I-ph<strong>on</strong>e<br />

and android applicati<strong>on</strong>”<br />

iii. How the market call for proposals are identified<br />

Very few of entrepreneurs stipulate through <strong>on</strong>line research, university publicati<strong>on</strong>s, and governmental<br />

institutes. Others entrepreneurs have no idea how to get the informati<strong>on</strong>.<br />

iv. The credibility, solvability and reliability of your actual clients<br />

Entrepreneurs have no idea where to get the needed informati<strong>on</strong><br />

v. How new financial aids for SMEs are identified<br />

Very few of entrepreneurs know to get financial aids for SMEs. Through Bank reviews, that usually<br />

provides good data especially about SME according to some entrepreneurs. They stipulated also that it is<br />

identified through meeting different financial companies to discuss the best financing soluti<strong>on</strong> with low<br />

interest rate and that meet the entrepreneur required budget. Others they will get informati<strong>on</strong> from<br />

Business seminars, social media, newspaper advertisements, networking.<br />

vi. The new law <strong>on</strong> the actual markets<br />

The majority of the entrepreneurs d<strong>on</strong>’t have accurate informati<strong>on</strong> about the new law <strong>on</strong> the actual markets.<br />

But a very small number of entrepreneurs indicated how they will get the informati<strong>on</strong> as follows: “C<strong>on</strong>duct<br />

market research. Identify products <strong>on</strong> the market, both <strong>on</strong>line and in stores and note which companies<br />

make them and where they are sold; Developing a great prototype this which will be evolved to a<br />

professi<strong>on</strong>ally made product; C<strong>on</strong>nect with other entrepreneurs in my local area or <strong>on</strong>line to share<br />

informati<strong>on</strong>; Etc.”<br />

(N.I): Entrepreneur would like to keep up to date with new trends, new designers, new quality c<strong>on</strong>trol<br />

processes, new machinery that can fasten producti<strong>on</strong>s and new soluti<strong>on</strong> for saving while decorating and<br />

creating fashi<strong>on</strong>. As well as staying tuned to recycling movement. They also need to be acknowledged<br />

permanently with the latest technologies, analysis, researches and publicati<strong>on</strong>s regarding their business,<br />

follow up observati<strong>on</strong>s and c<strong>on</strong>sultati<strong>on</strong>s. And easy access to these researches. Entrepreneurs seek<br />

informati<strong>on</strong> about the market growth rate in Leban<strong>on</strong> for each sector of activity. Online c<strong>on</strong>sumer<br />

behaviour, merchants’ requirements, customer reporting needs. Tracking changes of the client need.<br />

C<strong>on</strong>tinuous review of the market trends and experience that we got, following up with suppliers and<br />

customers. And informati<strong>on</strong> about laws about entering the market and about getting out of the market. In<br />

additi<strong>on</strong> they look for ministries laws and standards and for SME financial aids from the EU or the UN.<br />

(I.C.M.T.): Entrepreneurs answers as follows:<br />

403


Maroun Jneid and Antoine Tannous<br />

“Internet, factories, designers and engineers experiences. Following up and analysing and studying every<br />

informati<strong>on</strong> regarding their business leading to a c<strong>on</strong>tinuous ameliorati<strong>on</strong> of their skills <strong>on</strong> the acquired<br />

markets; Reading and surfing for new studies made by universities, companies, chemical experts in the<br />

domain and implementing a usage tracking system to detect the behaviour of the merchants and shoppers,<br />

as well follow up with randomly selected clients in order to take feedback. Open and participate to a forum<br />

to exchange informati<strong>on</strong>; Keep searching in the internet and other sources for new technologies and<br />

publicati<strong>on</strong>s. Also seeking informati<strong>on</strong> at banks and <str<strong>on</strong>g>European</str<strong>on</strong>g> Uni<strong>on</strong>’s or United Nati<strong>on</strong>s websites and<br />

offices. And by looking always to what the competitors are offering.<br />

to find the relevant partners for these tasks might not be easy in Leban<strong>on</strong>; Etc.”<br />

4. identificati<strong>on</strong> of the new qualified human resources<br />

(A.I.): According to the answers below. Entrepreneurs possess little informati<strong>on</strong> in this domain.<br />

The source for new individual skills<br />

Entrepreneur answers are as follow: “Universities, Training centers and associati<strong>on</strong>s, studies, training and<br />

c<strong>on</strong>tinuous training. Learn from previous experiences. Existing friends network, social media<br />

Entrepreneur would like to c<strong>on</strong>tact students of top universities in the Lebanese market. Also reputed<br />

recruitment agencies.”<br />

The new organizati<strong>on</strong> mode for producti<strong>on</strong><br />

Entrepreneurs have no accurate opini<strong>on</strong> about this topic<br />

Protecti<strong>on</strong> from the internal skills of the competitors<br />

Some of entrepreneurs said that the best way to protect the company form negative competiti<strong>on</strong> is to be<br />

ahead and strive for value producti<strong>on</strong> and excellence in the services delivery. Few entrepreneurs will offer<br />

b<strong>on</strong>uses according to criteria related to Grade, Quality, and Time<br />

Training and c<strong>on</strong>tinuous training<br />

Few entrepreneurs indicated that there are a variety of centers that can provide us with c<strong>on</strong>tinuing training<br />

and in the same company “Training is important to be having a learning organizati<strong>on</strong> and keeping the<br />

company up to date. The company should always update the team with periodical training sessi<strong>on</strong>s and<br />

seminars.”<br />

The labour law (employees’ status)<br />

Few of them said, “we will attend a workshop related to the topic. Or we will follow studies at university.<br />

And other said no deep understanding of labour law and others said that the full Lebanese labour is<br />

accessible <strong>on</strong> the internet and at the Labour Ministry.”<br />

(N.I): The majority of the entrepreneurs look for informati<strong>on</strong> about candidates regarding their educati<strong>on</strong><br />

levels, experiences and about salary range according to each activity field; some of them seek records<br />

about the best graduate from the local universities, ministries and specialized agencies in Leban<strong>on</strong> and the<br />

best legal status about the candidate. And how to put job announcement in newspapers<br />

(I.C.M.T.): Entrepreneurs will c<strong>on</strong>duct behavioural interview and CV discussi<strong>on</strong>. Others will make<br />

interviews, analysis and then they will decide. Some of them didn’t specify their needs. And some of them<br />

said they will ask third parties, friends, <strong>on</strong>line search, c<strong>on</strong>tact recruitment agencies, universities and head<br />

hunters. And another would like to create a wide professi<strong>on</strong>al network<br />

5. partnership possibility with other companies<br />

(A.I.): The majority of the entrepreneurs search for informati<strong>on</strong> regarding partnership. They knew that it is<br />

useful because they share revenues, experience and skills<br />

(N.I): Some of them ignore how to benefit from partnership. Others seek informati<strong>on</strong> about local and foreign<br />

companies: “The terms of partnership and the legal status and structure; The technological needs; Etc.”<br />

(I.C.M.T): Some entrepreneurs do not specify their needs while others indicated them as follows: “Meeting<br />

with potential partners; Universities, Research and development labs; Etc.”<br />

6. partnership possibility with<br />

(A.I.): The majority of the entrepreneurs do not specify their needs regarding research and development<br />

labs, universities and instituti<strong>on</strong>s and c<strong>on</strong>sulting companies. Some of them indicate that partnership with<br />

universities and instituti<strong>on</strong>s includes internships of students. And c<strong>on</strong>sulting companies help them through<br />

training, planning, coaching, assessments, market study<br />

(N.I): The majority of the entrepreneurs ignore what to answer. Few of them need to know the added value<br />

given to their business and type of agreements they should sign and what would be the fees. Also related<br />

laws<br />

(I.C.M.T.): Some of entrepreneur said that they will ask experienced people, universities, studies agencies,<br />

c<strong>on</strong>sulting companies and research, <strong>on</strong>line search, yellow pages, lawyers, audit experts and development<br />

labs<br />

7. foreign market opportunities<br />

(A.I.):<br />

Possibility of foreign market entry<br />

The majority of resp<strong>on</strong>dents d<strong>on</strong>’t have clear idea about foreign markets. Few of them indicated Germany,<br />

Dubai and Abu Dhabi<br />

Potential foreign market<br />

The majority of resp<strong>on</strong>dents d<strong>on</strong>’t know precisely which foreign market could be potential for them. Few of<br />

them stipulated the following countries: Egypt, Jordan and all the Arabic golf countries. USA, Germany,<br />

Canada, USA, India, UK, Netherlands, Sweden. The majority of the entrepreneurs are aware that Leban<strong>on</strong><br />

404


Maroun Jneid and Antoine Tannous<br />

is a small market, so foreign market is essential in order to grow up<br />

Call for proposal identificati<strong>on</strong> in foreign market<br />

The proposal stage is in advance for entrepreneurs. They are not able to answer to this questi<strong>on</strong> because<br />

they got no informati<strong>on</strong> about it<br />

(N.I): The informati<strong>on</strong> needed about foreign market opportunities were as follows: “Competiti<strong>on</strong>s,<br />

advertising and marketing companies’ c<strong>on</strong>diti<strong>on</strong>s and regulati<strong>on</strong>s, financial system of the countries;<br />

Markets that have the highest potential of customers; All the actors and factors, which could influence our<br />

decisi<strong>on</strong> making and the supply and the demand; The market size and the size of the populati<strong>on</strong>, tourists<br />

and their incomes and intenti<strong>on</strong> to purchase.”<br />

(I.C.M.T.): The majority of the entrepreneurs d<strong>on</strong>’t know how to get informati<strong>on</strong> about the foreign markets.<br />

Some entrepreneurs will seek informati<strong>on</strong> from c<strong>on</strong>sulting companies, traders, studies, <strong>on</strong>-line researches,<br />

mobile market research, market research, calling the embassies and asking friends, partners and c<strong>on</strong>tacts.<br />

Law c<strong>on</strong>sulting companies. Ministries<br />

8. ameliorati<strong>on</strong> of skills regarding the external market<br />

(A.I.): Need permanent studies about the external market, which the ameliorati<strong>on</strong> of skills is highly<br />

demanded through c<strong>on</strong>ferences, c<strong>on</strong>sulting, and analysis. Entrepreneurs said that a lot of informati<strong>on</strong> is<br />

available <strong>on</strong> the internet. Without being able to specify available informati<strong>on</strong> about product technologies,<br />

quality and promoti<strong>on</strong> methods<br />

(N.I): Entrepreneurs answers were as follow:<br />

“C<strong>on</strong>sulting & Researches, Analysis of the external market, market size, potential segments. Where, When,<br />

How and who to promote. Cost of TV ads in foreign countries; Latest technologies in science journals and<br />

publicati<strong>on</strong>s, latest results from research institutes, graduates from universities with degrees relevant to our<br />

business; How best they are recruiting qualified staff? What would be their main strengths? Etc.”<br />

(I.C.M.T.): “Direct statistics, research, Market research and intelligence. Searching for a local<br />

representative to get all required info. C<strong>on</strong>sultancy; Through following specialized blogs, websites and<br />

companies (mobile games and technology); and through downloading and testing existing games; Etc.”<br />

9. adaptati<strong>on</strong> c<strong>on</strong>straints <strong>on</strong> the exportati<strong>on</strong> markets regarding competitors<br />

(A.I.): Strategic and political c<strong>on</strong>straints, Operati<strong>on</strong>al c<strong>on</strong>straints, Regulatory c<strong>on</strong>straints, c<strong>on</strong>tractual<br />

c<strong>on</strong>straints and the potential of product. Languages adaptati<strong>on</strong> according the country. Another means to<br />

assess company's potential in exporting is by examining the unique or important features of products. If<br />

those features are hard to duplicate abroad, then it is likely that this will be successful overseas. To know if<br />

the product is patented and cannot be duplicated<br />

(N.I): “Which products should be selected for export development? What modificati<strong>on</strong>s, if any, must be<br />

made to adapt them for overseas markets regarding the competiti<strong>on</strong>? In each country, what is the basic<br />

customer profile of our competiti<strong>on</strong>? What marketing and distributi<strong>on</strong> channels are used by the competiti<strong>on</strong><br />

and what should be used to reach our customers? Etc.”<br />

(I.C.M.T.): Some entrepreneurs state that they build first a mini project in the foreign country, and then they<br />

observe how it works. Followed up by an ec<strong>on</strong>omical analysis. If it works they will do a direct investment.<br />

Others they will do <strong>on</strong>line deep internet searches, they will c<strong>on</strong>sult agencies, market study in the foreign<br />

country, sales office representative embassies, and ministries of foreign countries. And also the <str<strong>on</strong>g>European</str<strong>on</strong>g><br />

Uni<strong>on</strong> Office<br />

10. identificati<strong>on</strong> of competitors<br />

(A.I.):<br />

Competitor identity (Locals, Numbers, Volume)<br />

Entrepreneurs have little informati<strong>on</strong> about their competitors’ identity. They knew <strong>on</strong>ly their names and<br />

ignore other informati<strong>on</strong> related to their volume, turnover and profitability.<br />

New patents<br />

Few of entrepreneurs have informati<strong>on</strong> about products, services and prices of their competitors. The same<br />

answers appear c<strong>on</strong>cerning the competitor products’ technology and their methods of promoti<strong>on</strong>. Also the<br />

deadlines distributors of competiti<strong>on</strong>, as well as <strong>on</strong> their patents<br />

(N.I): The majority of the entrepreneurs search for informati<strong>on</strong> about the competiti<strong>on</strong>s size, market share<br />

and details about their offers: products, services, technology, prices, distributi<strong>on</strong> channels and promoti<strong>on</strong>al<br />

campaign. Also, their strengths, weakness, objectives, strategies and their c<strong>on</strong>sumers’ behaviours<br />

(I.C.M.T.): The majority of entrepreneurs will get informati<strong>on</strong> from surveys, market research, <strong>on</strong>line search,<br />

c<strong>on</strong>sulting. They will also c<strong>on</strong>tact some friends in different local market research companies and will follow<br />

the progress of the competitors and gather informati<strong>on</strong> about their clients. Others will get informati<strong>on</strong> from<br />

the industrial or commercial register<br />

11. competitor functi<strong>on</strong>al strength and weakness<br />

(A.I.): Entrepreneurs have very few informati<strong>on</strong> about their competitors.<br />

(N.I): Entrepreneurs need to know the strategy of their competitor, their capacity of producti<strong>on</strong>,<br />

organizati<strong>on</strong>, financial and human resources capabilities, and also to evaluate their markets. As well about<br />

new entrants, competitors’ fusi<strong>on</strong>s, acquisiti<strong>on</strong>s and the main features and services that return high<br />

income. Others require sales and profits by market and brand, market shares and size, distributi<strong>on</strong><br />

systems, profile of senior management, advertising strategy and spending, c<strong>on</strong>sumer profile and attitude<br />

405


Maroun Jneid and Antoine Tannous<br />

furthermore Customer retenti<strong>on</strong> levels<br />

(I.C.M.T.): Same answer as for topic 10, plus majority of the entrepreneurs need to search informati<strong>on</strong><br />

about their competitors and few of them stipulate that they will buy competitors’ products and analysing<br />

them.<br />

Other entrepreneurs will collect their needed informati<strong>on</strong> as follow:<br />

“Recorded Data: annual report & accounts, press releases, newspaper articles, analysts’ reports,<br />

regulatory reports, government reports, presentati<strong>on</strong>s and speeches; Observable Data: price lists,<br />

advertising campaigns, promoti<strong>on</strong>s, tenders, patent applicati<strong>on</strong>s; Opportunistic Data: focus group, meetings<br />

with suppliers, trade shows, sales force meetings, seminars, c<strong>on</strong>ferences, recruiting ex-employees,<br />

discussi<strong>on</strong> with shared distributors, social c<strong>on</strong>tacts with competitors”<br />

12. adaptati<strong>on</strong> capacity regarding envir<strong>on</strong>mental change<br />

(A.I.): Pers<strong>on</strong>al experience. C<strong>on</strong>sultancy. Developing partners with universities<br />

(N.I): What is the time to adapt and what is the cost? Entrepreneurs need also to know the culture changes<br />

and the technology changes in order to adapt with the changes<br />

(I.C.M.T.): C<strong>on</strong>sultancy and checking statistics about all the markets. Annual report & accounts, press<br />

releases, newspaper articles, analysts’ reports, regulatory reports, government reports, presentati<strong>on</strong>s and<br />

speeches<br />

13. latest informati<strong>on</strong> and communicati<strong>on</strong> technologies<br />

(A.I.):<br />

Informati<strong>on</strong> collecti<strong>on</strong> tools (Internet, database, …):<br />

“Using the smarter technologies like, CRM, IOS, Android, windows ph<strong>on</strong>e, internet, external databases,<br />

and third parties. Using client server apps Etc.”. Other entrepreneurs’ specified, “the pers<strong>on</strong>al interviews,<br />

Self-administered surveys, focus groups, internet, media outlets, questi<strong>on</strong>naires, data bases, press,<br />

exhibiti<strong>on</strong>s, c<strong>on</strong>tacts, etc.”<br />

Informati<strong>on</strong> treatment tools:<br />

Some entrepreneurs added “Semantic, linguistic, statistic and neur<strong>on</strong>al networking methods”<br />

Informati<strong>on</strong> storage (database, documentary database):<br />

Some entrepreneurs added “(database, documentary database), Etc.”<br />

Informati<strong>on</strong> diffusi<strong>on</strong> tools:<br />

Some entrepreneurs added “mailchimp, google companies”<br />

(N.I): The latest methods in informati<strong>on</strong> collecti<strong>on</strong>, informati<strong>on</strong> treatment, informati<strong>on</strong> storage and<br />

informati<strong>on</strong> diffusi<strong>on</strong>, and the capability to implement, store and use the informati<strong>on</strong> in a running business.<br />

New changes in communicati<strong>on</strong> protocols, the new databases, the latest technological advances for the<br />

web and the mobile, mainly in web and security of e-commerce and the new programming languages. As<br />

well, the internet growth and adaptability, cost and performance, this includes .net framework latest<br />

versi<strong>on</strong>s, mailing systems, ph<strong>on</strong>e applicati<strong>on</strong>s<br />

(I.C.M.T.): Using internet websites, related blogs, newsletters, ministry of telecommunicati<strong>on</strong>, visiting<br />

expositi<strong>on</strong>s, reading magazines c<strong>on</strong>cerning the smarter technologies. Signing up in news and professi<strong>on</strong>al<br />

sites for close follow-up of the technologies progress. Taking special courses and training. Taking advices<br />

from friends. Then Building first a mini project followed by an ec<strong>on</strong>omical observati<strong>on</strong>s and analysis<br />

7. C<strong>on</strong>clusi<strong>on</strong><br />

From this qualitative study we can c<strong>on</strong>clude the following about the entrepreneurs of SMEs including<br />

the entrepreneurs in the ICT sector:<br />

A lack of objectivity and absence of market research for some entrepreneurs, in their choice of<br />

target markets<br />

A lack of awareness about the importance of an informati<strong>on</strong> system support to face the<br />

informati<strong>on</strong> evaluati<strong>on</strong> challenges<br />

A lack of informati<strong>on</strong> about the strategy of the competitors “potential markets, producti<strong>on</strong>,<br />

organizati<strong>on</strong>, financial and human resources capabilities” and weaknesses in identifying the<br />

source of the informati<strong>on</strong> related to the ameliorati<strong>on</strong> of skills <strong>on</strong> the acquired market in their earlystage<br />

A need for a c<strong>on</strong>tinuous update about the latest regulati<strong>on</strong>s, technologies, analysis, researches<br />

and publicati<strong>on</strong>s regarding their business from surveys, market research, <strong>on</strong>line search,<br />

c<strong>on</strong>sulting about competitors<br />

The importance of a market behaviour Informati<strong>on</strong> Tracking System for the survival<br />

Product and Language adaptati<strong>on</strong> c<strong>on</strong>straints <strong>on</strong> the exportati<strong>on</strong> markets regarding competitors<br />

The results of this study highlight the important role of the Informati<strong>on</strong> Systems in the implementati<strong>on</strong><br />

and development of the entrepreneur business and as Informati<strong>on</strong> technologies have brought more<br />

406


Maroun Jneid and Antoine Tannous<br />

visibility to strategic management, more entrepreneurs have gained awareness of their survival<br />

c<strong>on</strong>diti<strong>on</strong>s and competitive positi<strong>on</strong>ing.<br />

There is also a need to create awareness about successful entrepreneurial process and Competitive<br />

Intelligence software soluti<strong>on</strong>s supporting the entrepreneurial activities since the early-stage; while<br />

existing soluti<strong>on</strong>s are mostly designed for large-scaled enterprises, SMEs require CI soluti<strong>on</strong> with light<br />

architectures that c<strong>on</strong>siders also the ec<strong>on</strong>omic factors (language, source of informati<strong>on</strong>, lack of skilled<br />

human resources in this domain) where the entrepreneurial process is taken place.<br />

For that an affordable software soluti<strong>on</strong> architecture that can be integrated by SMEs since their earlystage<br />

can profit from the advantages of a Service Oriented Architecture deployed in a cloud<br />

computing (SaaS, PaaS and IaaS) service envir<strong>on</strong>ment.<br />

References<br />

BMI, 2012. Leban<strong>on</strong> Informati<strong>on</strong> Technology Report, Business M<strong>on</strong>itor Internati<strong>on</strong>al.<br />

Bygrave, W.D., (2009). The Entrepreneurial Process. In The Portable MBA in Entrepreneurship. Hoboken, NJ,<br />

USA: John Wiley & S<strong>on</strong>s.<br />

Dutta, S. & Mia, I., (2011). The Global Informati<strong>on</strong> Technology Report 2010–2011, Geneva, Switzerland: World<br />

Ec<strong>on</strong>omic Forum & INSEAD.<br />

Grabova, O. et al., (2010). Business Intelligence for Small and Middle-Sized Entreprises. SIGMOD Record, 39(2).<br />

Hohhof, B., (1994). Developing informati<strong>on</strong> systems for competitive intelligence support. Library Trends, 43,<br />

pp.226–238.<br />

Hussein, S. & Salles, M., (2003). Une Classificati<strong>on</strong> des besoins en intelligence éc<strong>on</strong>omique Exprimés par les<br />

dirigeants des PME. In IERA (Intelligence Ec<strong>on</strong>omique : Recherches et Applicati<strong>on</strong>). Nancy, France.<br />

IDRC, (2010). Global Entrepreneurship M<strong>on</strong>itor: Rapport Régi<strong>on</strong>al GEM-MOAN 2009 (Moyen-Orient et Afrique<br />

du Nord), Internati<strong>on</strong>al Development Research Center.<br />

Porter, M.E., (1980). Competitive strategy: techniques for analyzing industries and competitors. New York: Free<br />

Press.<br />

Schwab, K., (2011). The Global Competitiveness Report 2011–2012, Geneva, Switzerland: World Ec<strong>on</strong>omic<br />

Forum.<br />

Team SITE, (2007). Modeling and Development of Ec<strong>on</strong>omic Intelligence Systems, Activity report, INRIA.<br />

407


Method Engineering Approach to the Adopti<strong>on</strong> of<br />

Informati<strong>on</strong> Technology Governance, Risk and Compliance<br />

in Swiss Hospitals<br />

Mike Krey 1 , Steven Furnell 1 , Bettina Harriehausen 2 and Matthias Knoll 2<br />

1 Centre for Security, Communicati<strong>on</strong>s & Network Research - Plymouth<br />

University, Plymouth, UK<br />

2 Department of Computer Sciences; Faculty of Ec<strong>on</strong>omics and Business<br />

Administrati<strong>on</strong> - Darmstadt University of Applied Sciences, Darmstadt,<br />

Germany<br />

Mike.Krey@zhaw.ch<br />

Abstract: Against the background of the current reforms and an aftermath of increasing regulati<strong>on</strong> in the health<br />

care sector, hospitals enhance and integrate c<strong>on</strong>cepts of IT Governance, Risk and Compliance (IT GRC). Based<br />

<strong>on</strong> experiences with isolated and often immature partial c<strong>on</strong>cepts in these fields, the major challenges for the<br />

adopti<strong>on</strong> of IT GRC in hospitals are close-meshed organisati<strong>on</strong>al structures, legal restraints and over the years<br />

increased heterogeneous IT systems, which are just a few aspects that make hospitals a sensible field for the<br />

implementati<strong>on</strong> and governance of IT. In this paper a method that supports the adopti<strong>on</strong> of integrated IT GRC<br />

c<strong>on</strong>cepts is been developed. The proposed method is comprised of different method elements that support the<br />

relevant c<strong>on</strong>ceptual, organisati<strong>on</strong>al, technical, and cultural aspects of the hospital envir<strong>on</strong>ment.<br />

Keywords: method, design science research, health care, IT governance, risk, compliance<br />

1. Introducti<strong>on</strong><br />

As many developed countries, Switzerland is facing increasing expenditures for their health system:<br />

11.4% of the Swiss gross domestic product (GDP) in 2009 has been spent <strong>on</strong> health care – the<br />

OECD average was 9.6%. The Swiss health system is characterised by three key features: the first is<br />

its federal structure and a complex system of powers and resp<strong>on</strong>sibilities by level of government; the<br />

sec<strong>on</strong>d is a political traditi<strong>on</strong> of direct democracy and governance through c<strong>on</strong>sensus; and the third is<br />

an emphasis <strong>on</strong> managed competiti<strong>on</strong> within the health system rather than state-run (e.g. integrated<br />

health care model) arrangements. Legislative reform efforts therefore primarily aim at increasing<br />

productivity within health care, while ensuring the quality of care. Central catalyser of these changes<br />

is the setup of incentives in in-patient care by diagnostic related groups (DRGs) in many <str<strong>on</strong>g>European</str<strong>on</strong>g><br />

countries. Since 2009, hospitals in Switzerland have been transiti<strong>on</strong>ing to a new remunerati<strong>on</strong><br />

approach which provides case-based payments. This new system is been introduced in 2012 and is<br />

becoming the dominant payment mechanism for hospitals in Switzerland. As in many other countries<br />

which have already introduced DRGs (e.g., Germany and Austria) the new tariff system promises<br />

transparency and comparability of in-patient services across hospitals. For the affected hospitals it is<br />

necessary to develop c<strong>on</strong>cepts and reforms to work more efficient and have c<strong>on</strong>trol of their business<br />

(medical and administrative) processes. As in many other industry sectors, when it comes to<br />

optimisati<strong>on</strong> and reorganisati<strong>on</strong> of processes, informati<strong>on</strong> technology (IT) can be an effective<br />

instrument for improvements (Berensmann 2005). IT can be a driver for diversificati<strong>on</strong> in competiti<strong>on</strong><br />

and creati<strong>on</strong> of innovative strategic competitive advantages (Piccoli and Ives 2005). This endeavour<br />

requires besides a vast understanding of the related business processes and the ability to deal with<br />

innovati<strong>on</strong>, a goal-oriented effort of given resources. This calls for IT governance in terms of an<br />

integrated and comprehensive approach to effectively align, invest, measure, deploy and sustain the<br />

strategic and tactical directi<strong>on</strong> and value propositi<strong>on</strong> of IT in support of the business with respect to<br />

the management of risk and compliance issues (IT GRC) (Weill and Ross 2004).<br />

1.1 Problem statement and objectives<br />

Although the paper at hand focuses <strong>on</strong> the Swiss health system it may be applicable to different<br />

countries which share the same problems. In additi<strong>on</strong> to the given fact of an aging society other<br />

challenges to be met by the endeavour to fundamental reorganisati<strong>on</strong> of the health care sector are,<br />

e.g. close-meshed organisati<strong>on</strong>al structures within the hospital and between its stakeholders, legal<br />

restraints based <strong>on</strong> a c<strong>on</strong>current federal and decentralised structure and over the years increased<br />

heterogeneous IT systems (VIG, 2005) which are just a few aspects that make the health care sector<br />

a sensible field for the implementati<strong>on</strong> and governance of IT (Porter and Teisberg 2004). In the last<br />

408


Mike Krey et al.<br />

years a range of best practice models (e.g., C<strong>on</strong>trol Objectives for Informati<strong>on</strong> and Related<br />

Technology (COBIT) or Informati<strong>on</strong> Technology Infrastructure Library (ITIL)) as well as proprietary<br />

frameworks have been developed (e.g., Microsoft Operati<strong>on</strong>s Framework (MOF)). However, these<br />

frameworks promises that they can be implemented independent from the size, industry sector and<br />

status of IT innovati<strong>on</strong> of the enterprise (Johannsen et al. 2007). Representative studies which have<br />

been c<strong>on</strong>ducted in both, Switzerland and internati<strong>on</strong>al let assume that the c<strong>on</strong>cepts and models<br />

meeting the multi-layered challenges in hospitals so far can be classified as inadequate and n<strong>on</strong>sustainable<br />

(Hoerbst et al. 2011; Krey et al. 2010; Koebler et al. 2010). Reas<strong>on</strong>s for a less<br />

comprehensive spread of best practices in the hospital envir<strong>on</strong>ment are as varied as the challenges<br />

do deal with. In the case of COBIT the lengthy documentati<strong>on</strong> which c<strong>on</strong>sists of several hundreds of<br />

pages can be seen as an obstacle therefore plain and simple ask too much of IT executives or the<br />

circumstance that a COBIT implementati<strong>on</strong> approach starts from the premise that a basic<br />

understanding of IT governance principles, issues and presuppositi<strong>on</strong>s like an IT strategy, corporate<br />

goals or a process-oriented IT department are already defined or still in place. According to Krey et<br />

al., an inadequate maturity of these issues right up to a complete lack however can be found in Swiss<br />

hospitals which make a “<strong>on</strong>e size fits all” approach questi<strong>on</strong>able and calls for an anticipative and more<br />

fundamental approach to the implementati<strong>on</strong> of IT GRC (2010). The main objective of this research<br />

work is therefore to develop a design artifact for the adopti<strong>on</strong> of IT GRC in hospitals taking domain<br />

specific requirements, limitati<strong>on</strong>s and c<strong>on</strong>siderati<strong>on</strong>s into account. Based <strong>on</strong> the previous problem<br />

statement the following research questi<strong>on</strong> is addressed:<br />

“How can the adopti<strong>on</strong> of IT GRC in hospitals, with respect to domain-specific<br />

characteristics and requirements, be systematically supported?”<br />

The paper at hand is structured as follows: In the sec<strong>on</strong>d secti<strong>on</strong> the methodology and research<br />

framework are presented. In Secti<strong>on</strong> 3 related work <strong>on</strong> the definiti<strong>on</strong> and implementati<strong>on</strong> of IT GRC<br />

are discussed. The related domain-specific requirements for the adopti<strong>on</strong> of IT GRC are presented in<br />

Secti<strong>on</strong> 4. Secti<strong>on</strong> 5 discusses the derived meta-model of the method and its method elements. In the<br />

final secti<strong>on</strong>, c<strong>on</strong>clusi<strong>on</strong>s are drawn and an outlook <strong>on</strong> further research is given.<br />

2. Methodology and research framework<br />

The domain-specific problem and the thereof derived objectives are assigned to the research<br />

discipline of informati<strong>on</strong> systems (IS). Although IS has been affected by different sciences, a comm<strong>on</strong><br />

opini<strong>on</strong> about adequate research methods to be used, is still missing. Hence, two research paradigms<br />

are currently discussed in IS: 1) behaviour-oriented research (BS) and 2) design science research<br />

(DSR). Whereas BS aims at the descripti<strong>on</strong> and predicti<strong>on</strong> of phenomena through the applicati<strong>on</strong> of<br />

appropriate theories, DSR aims basically at the understanding of the truth in relati<strong>on</strong> to a natural<br />

object of observati<strong>on</strong>, and causes artificial changes <strong>on</strong> the research object itself (March and Smith<br />

1995). March and Smith have proposed a list of four general artifacts which can be classified in terms<br />

of DSR output: 1) c<strong>on</strong>structs, 2) models, 3) methods, and 4) instantiati<strong>on</strong>s (p. 256). Methods describe<br />

a number of activities (e.g., guidelines or algorithms) used to carry out tasks or develop findings.<br />

“Methods are goal directed plans for manipulating c<strong>on</strong>structs so that the soluti<strong>on</strong> statement model is<br />

realised” (March and Smith 1995). Because of their problem-solving feature, methods are c<strong>on</strong>sidered<br />

as an appropriate artifact of this research work. The method to be developed is thereof the artifact of<br />

the development process in terms of DSR and is been understood as a systematic guidance, which<br />

describes in a comprehensible way how to address IT GRC principles c<strong>on</strong>sidering possibilities and<br />

c<strong>on</strong>straints in hospitals. In order to systemise the DSR development process, the approach by Hevner<br />

et al. (2004) has been applied. The following phases and their purposes have been identified for this<br />

research work (cf. Figure 1). Within the (1) “problem identificati<strong>on</strong>” phase, the weaknesses of existing<br />

approaches and the thereby arising possibilities and goals of new artifacts are derived. In this work,<br />

the lack of disseminati<strong>on</strong> of IT GRC methods in the health care sector is the initial problem. The<br />

research gap is characterised by the lack of methods and practices that meet the identified<br />

characteristics and requirements. Phase (2) includes the “requirements analysis” and the review of<br />

alternative soluti<strong>on</strong>s. The actual development of the artifact which includes the definiti<strong>on</strong> of related<br />

elements and the rigor applicati<strong>on</strong> of research methods are part of the third phase (3) “method<br />

development”. The testing of the proposed method aims to point out its utility through suitable domain<br />

experts (hospital executives) and is part of the “evaluati<strong>on</strong>” phase (4). The (5) “communicati<strong>on</strong> and<br />

reflexi<strong>on</strong>” of the research results towards hospital executives is already part of the phases “problem<br />

identificati<strong>on</strong>”, “method development” and “evaluati<strong>on</strong>”. Towards the science community the<br />

communicati<strong>on</strong> and reflexi<strong>on</strong> of the results takes place through the publishing of academic papers.<br />

The “envir<strong>on</strong>ment” describes the subject area of research in the dimensi<strong>on</strong>s people, (business)<br />

409


Mike Krey et al.<br />

organisati<strong>on</strong> and technology. In the research work at hand the envir<strong>on</strong>ment is represented through<br />

the hospital and its particularities (cf. Secti<strong>on</strong> 4). The “knowledge base” provides the underpinnings<br />

and methodologies for the implementati<strong>on</strong> of the research such as the principles of method<br />

engineering (ME) (cf. Secti<strong>on</strong> 5).<br />

Figure 1: Overall research process, according to Hevner et al. 2004<br />

The following secti<strong>on</strong> aims at the investigati<strong>on</strong> of existing approaches and reflects therefore <strong>on</strong> the<br />

first research phase "problem identificati<strong>on</strong>” (cf. Figure 1). The review <strong>on</strong> the different IT GRC areas<br />

has been c<strong>on</strong>ducted with regard to their ability to provide methodical support for the adopti<strong>on</strong> of IT<br />

GRC c<strong>on</strong>cepts.<br />

3. IT governance, risk and compliance – a literature review<br />

In this research work the GRC c<strong>on</strong>text means IT related governance and describes the topics and<br />

methods that IT executives needs to address to govern IT within their hospital under c<strong>on</strong>siderati<strong>on</strong> of<br />

imminent risks and legal requirements. As stated by PWC (2006) “GRC is not new. As individual<br />

issues, governance, risk management and compliance have always been fundamental c<strong>on</strong>cerns of<br />

business and its leaders. What is new is an emerging percepti<strong>on</strong> of GRC as an integrated set of<br />

c<strong>on</strong>cepts that, when applied holistically within an organisati<strong>on</strong> can add significant value and provide<br />

competitive advantage”. Driven by the current ec<strong>on</strong>omic situati<strong>on</strong>, an integrated approach to IT GRC<br />

is emerging rapidly (Gill and Purushottam 2008). This organisati<strong>on</strong>-wide approach “help all<br />

stakeholders collaborate effectively, reduce overall business risk, ensure better compliance and<br />

establish competitive advantage in the market place” (Gill and Purushottam 2008, p. 38). Within the<br />

work by Gericke et al. (2009) 21 method fragments have been identified for the support of GRC<br />

soluti<strong>on</strong>s in large companies. These method fragments have been classified into c<strong>on</strong>ceptual,<br />

strategic, organisati<strong>on</strong>al, technical and cultural aspects and finally roles have been defined and<br />

assigned to corresp<strong>on</strong>ding method fragments. Although each method fragment describes important<br />

activities, techniques and results, they lack of a defined process model in order to specify the<br />

sequence of activities needed to maintain the GRC system. Furthermore, as part of the development<br />

process of situati<strong>on</strong>al methods, Mirbel and Ralyté (2006) claimed for a derivati<strong>on</strong> of rules for the<br />

assembly of method fragments (rule base). The approach by Gericke et al. (2009) did not meet this<br />

obligati<strong>on</strong> and makes it still an open issue. In the following secti<strong>on</strong>, the different elements of IT GRC<br />

found in literature are examined. This step can be understood as part of requirements analysis<br />

described in Chapter 4.<br />

410


3.1 IT governance<br />

Mike Krey et al.<br />

IT governance is recognised as an integral part of enterprise governance today. It “c<strong>on</strong>sists of the<br />

leadership and organisati<strong>on</strong>al structures and processes that ensure that the organisati<strong>on</strong>’s IT sustains<br />

and extends the organisati<strong>on</strong>’s strategies and objectives” (IT Governance Institute 2003, p.10). IT<br />

governance has been defined in various ways (e.g., Haes and Grembergen, 2008, Weill and Ross,<br />

2004). These definiti<strong>on</strong>s differ in some aspects; however, they focus <strong>on</strong> the same issues, such as<br />

including management oversight, processes and rules for c<strong>on</strong>ducting various activities and a<br />

measurement and reporting mechanism of status and quality. However, a variety of c<strong>on</strong>cepts have<br />

been developed in the field of IT governance and value-based management (e.g. Arnold and Davies<br />

2000; Strenger 2004) without elaborating <strong>on</strong> implementati<strong>on</strong> or adopti<strong>on</strong> issues. COBIT is <strong>on</strong>e of<br />

these c<strong>on</strong>cepts and seeks to ensure that IT resources are aligned with an organisati<strong>on</strong>’s business<br />

objectives in order to balance IT risk and returns. COBIT has been developed based <strong>on</strong> practical<br />

experiences from business which have been c<strong>on</strong>solidated to generally accepted rules, processes,<br />

and characteristics. While a few method elements can be found in the COBIT framework (process,<br />

activity, role, goal, result, metric etc.) it lacks of a defined underpinning meta-model in order to present<br />

the logical and semantic relati<strong>on</strong>ships between the elements. The existing structures and tools within<br />

the framework make it a holistic c<strong>on</strong>cept, which provides the user navigati<strong>on</strong> through the different<br />

aspects but at the same time hamper the integrati<strong>on</strong> of new framework elements or organisati<strong>on</strong><br />

specific issues. Such an approach was even performed in the aftermath by Looso (2010).<br />

3.2 IT risk management<br />

In the field of IT risk management a view c<strong>on</strong>tributi<strong>on</strong>s can be found dealing with risk management<br />

frameworks which allow for the identificati<strong>on</strong> of and resp<strong>on</strong>se to risks in organisati<strong>on</strong>s (e.g., Jallow et<br />

al. 2007; Steinberg et al. 2004). Well known frameworks are, for example, the COSO framework or<br />

the risk management framework of the Software Engineering Institute (SEI). However, <strong>on</strong>ly few<br />

sources (e.g., Bruehwiler 2003) address the implementati<strong>on</strong> of such risks management frameworks<br />

and the challenges to overcome. These sources provide some valuable instructi<strong>on</strong>s, but lack of a<br />

structured and methodical c<strong>on</strong>tributi<strong>on</strong> to the problem-solving.<br />

3.3 IT compliance<br />

Recent regulati<strong>on</strong>s such as SOX, Basel II or Solvency II have driven research in the field of<br />

compliance management. Such research usually focuses <strong>on</strong> the identificati<strong>on</strong> of appropriate c<strong>on</strong>trols<br />

(Proctor, 2005) or <strong>on</strong> instructi<strong>on</strong>s <strong>on</strong> how to integrate c<strong>on</strong>trols into business processes (Rausch,<br />

2006). However, <strong>on</strong>ly little advice is given for the integrated implementati<strong>on</strong> of appropriate compliance<br />

c<strong>on</strong>cepts. Sharing of sensitive health informati<strong>on</strong> has been a c<strong>on</strong>cern for regulators for years.<br />

Different approaches have been taken: the U.S., e.g. relies <strong>on</strong> a sectoral regulati<strong>on</strong>, which specifically<br />

protects health informati<strong>on</strong> through legislati<strong>on</strong>s such as the Health Insurance Portability and<br />

Accountability Act of 1996 (HIPAA) (USDHHS 2003). The HIPAA is a regulative framework that<br />

includes am<strong>on</strong>g other things a privacy standard as well as a security standard. Covered organisati<strong>on</strong>s<br />

are health care providers, health plans, health care clearinghouses, as well as their business<br />

associates. However, many hospitals in Switzerland are owned or partly financed by the government;<br />

a stricter regulati<strong>on</strong> applies for them. Data is protected in a similar manner as in the U.S., it is<br />

distinguished between sensitive data and profiles and n<strong>on</strong>-sensitive data (EDOEB 2012) However,<br />

the Swiss health system is based <strong>on</strong> a federal and decentralised structure at the same time. The<br />

twenty-six cant<strong>on</strong>s in Switzerland are resp<strong>on</strong>sible to implement federal laws and ordinances, provide<br />

health care and cover parts of the hospitals costs (Walter 2008). The high degree of cant<strong>on</strong>s<br />

independence enables them to supplemented federal regulati<strong>on</strong>s by cant<strong>on</strong>al legislati<strong>on</strong>. This partial<br />

aut<strong>on</strong>omy has led to a great fragmentati<strong>on</strong> with slightly different health systems across Switzerland<br />

and eventuated in twenty-six health departments which make additi<strong>on</strong>al co-ordinati<strong>on</strong> efforts between<br />

the federal and cant<strong>on</strong>al entities necessary. For that reas<strong>on</strong>, the health care sector in Switzerland<br />

needs to c<strong>on</strong>firm compliance of, e.g. IT policies, standards, procedures and methodologies with legal<br />

and regulatory requirements. Taking the characteristics of the IT GRC approaches and their<br />

methodical support into account, it becomes obvious that existing approaches from the disciples of<br />

SME and best practice do not suffice for the defined problem at hand. The following secti<strong>on</strong> picks up<br />

<strong>on</strong> the topic of tailored approaches to a given c<strong>on</strong>text and aims at the investigati<strong>on</strong> of the hospital<br />

envir<strong>on</strong>ment and IT GRC approaches and therefore reflects <strong>on</strong> the sec<strong>on</strong>d research phase<br />

"requirements analysis” (see Figure 1).<br />

411


4. Requirements analysis<br />

Mike Krey et al.<br />

Since the method in this research work does not seek for the design or reorganizati<strong>on</strong> of the hospital,<br />

its departments or its organizati<strong>on</strong>al principles directly, but rather aims at the support to the adopti<strong>on</strong><br />

of IT GRC, requirements in this work describe given circumstances, principles as well as limitati<strong>on</strong>s in<br />

order to ensure the completeness and c<strong>on</strong>sistency of the method as such and its elements as well as<br />

ensure the method`s applicability and acceptability. Based <strong>on</strong> these understandings requirements<br />

engineering (RE) can be described as a totality of all activities for “finding out, analysing, documenting<br />

and checking” requirements (Sommerville 2011, p.83). The use of the term "engineering" is intended<br />

here to imply the systematic applicati<strong>on</strong> of repeatable techniques to ensure that requirements have<br />

been elicited complete, c<strong>on</strong>sistent and with high c<strong>on</strong>text relevance. Therefore a structured approach<br />

following RE principles is crucial for this research work to guarantee <strong>on</strong> the <strong>on</strong>e hand the<br />

completeness and relevance of the requirements in terms of its purpose and the applicati<strong>on</strong> within the<br />

desired envir<strong>on</strong>ment and <strong>on</strong> the other hand to comply with the demand for scientific rigor of the whole<br />

research project (cf. Figure 1). Following the approach by Sommerville (2011) RE process of this<br />

research work c<strong>on</strong>sists of three high-level activities: (1) requirements elicitati<strong>on</strong>, (2) requirements<br />

specificati<strong>on</strong>, and requirements validati<strong>on</strong>.<br />

4.1 Requirements elicitati<strong>on</strong><br />

The (1) requirements elicitati<strong>on</strong> describes the discovering of the related characteristics and then<br />

taking this unstructured collecti<strong>on</strong> and organises them in to coherent clusters. In doing so, domainrelated<br />

restricti<strong>on</strong>s, influences as well as peculiarities of hospitals (cf. Figure 1 "market view") are<br />

identified and applied to various enablers and inhibitors to IT GRC approaches. The reviewed<br />

characteristics are building the basis, which might help to understand the challenges that a<br />

sustainable IT GRC adopti<strong>on</strong> has to meet within hospitals. The following list of characteristics is not<br />

exhausted but helps to understand the specific challenges to an IT GRC adopti<strong>on</strong> (cf. Table 1).<br />

Table 1: Characteristics of IT GRC approaches for health care<br />

Characteristic (C.xx) Descripti<strong>on</strong><br />

“A life of its own” can be found in different organisati<strong>on</strong>al units within health care<br />

organisati<strong>on</strong>s, as a result of the specialisati<strong>on</strong> and divisi<strong>on</strong> of labour, which are<br />

based <strong>on</strong> the complexity of medical services provided by physicians (e.g.,<br />

C.01 Aut<strong>on</strong>omy<br />

internal medicine, surgery, radiology, etc.). The aut<strong>on</strong>omy of organisati<strong>on</strong>al units<br />

lead c<strong>on</strong>sequently to decentralised decisi<strong>on</strong>-making, management of structures,<br />

informati<strong>on</strong> and authorities (Rockwell and Johns<strong>on</strong> 2005).<br />

The variety and complexity of IT GRC approaches relates to the several<br />

C.02 Variety and frameworks, reference models and best practices, issued by both internati<strong>on</strong>al<br />

complexity of IT GRC standardisati<strong>on</strong> organisati<strong>on</strong>s and private organisati<strong>on</strong>s exist in additi<strong>on</strong> to the<br />

approaches de facto standard COBIT for managing the different aspects of IT (Haes and<br />

Grembergen 2008).<br />

The CIO positi<strong>on</strong> within the organisati<strong>on</strong> must develop a trusted relati<strong>on</strong>ship with<br />

top management to succeed in this resp<strong>on</strong>sibility. The health care organisati<strong>on</strong><br />

is a political arena (Hoerbst et al. 2011). An IT department has relatively little<br />

C.03 Role of IT<br />

influence compared to other (medical) organisati<strong>on</strong>al units within the<br />

organisati<strong>on</strong>. With limited organisati<strong>on</strong>al influence, the IT department with the<br />

CIO must educate management <strong>on</strong> the necessity of an integrated IT architecture<br />

to avoid bounded rati<strong>on</strong>ality during IT development.<br />

4.2 Requirements specificati<strong>on</strong><br />

The (2) requirements specificati<strong>on</strong> describes the process of putting the various characteristics into a<br />

standard which guarantees the unambiguity, completeness and c<strong>on</strong>sistency of the derived<br />

requirements. This activity is taking the investigated relati<strong>on</strong>s between hospitals and its particularities<br />

and IT GRC approaches and their characteristics and analysis the requirements needed to the<br />

adopti<strong>on</strong> of IT GRC in the hospitals. As this research is assigned to the discipline of DSR, just a few<br />

c<strong>on</strong>tributi<strong>on</strong>s can be found dealing with the systematisati<strong>on</strong> and provisi<strong>on</strong> of patterns/principles<br />

supporting the actual method development process. The following requirements analysis is based <strong>on</strong><br />

the approach by Gericke (2009), who has expanded existing DSR patterns for the artifact c<strong>on</strong>structi<strong>on</strong><br />

by including patterns from engineering. The derivati<strong>on</strong> process c<strong>on</strong>sists of three steps and aims<br />

taking all described characteristics into account. In the first step (1), the given similarities and different<br />

levels of granularity of the identified characteristics will be balanced logically (generalisati<strong>on</strong>). The<br />

generalisati<strong>on</strong> is a comm<strong>on</strong> methodology in DSR, as it reduces <strong>on</strong> the <strong>on</strong>e hand the complexity and<br />

412


Mike Krey et al.<br />

number of relati<strong>on</strong>s between characteristics and guarantees <strong>on</strong> the other hand the validity and<br />

traceability of requirements through logically identifiable relati<strong>on</strong>ships. As a final outcome, a list core<br />

statement (categories) is obtained. Based <strong>on</strong> the core statements in the sec<strong>on</strong>d step (2) the central<br />

requirements can be derived. In the third step (3) it will be examined, if each characteristic is<br />

addressed through minimum <strong>on</strong>e requirement. However it is not mandatory that all characteristics of a<br />

category address the particular derived requirement. It is more important that all characteristics are<br />

c<strong>on</strong>sidered. As an output of this analysis the following list of requirements to a method for the<br />

adopti<strong>on</strong> of IT GRC within the hospital envir<strong>on</strong>ment is obtained (cf. Table 2).<br />

Table 2: Requirements to a method for the adopti<strong>on</strong> of IT GRC within the hospital envir<strong>on</strong>ment<br />

Requirement<br />

Descripti<strong>on</strong><br />

(R.xx)<br />

R.01 The method has to take existing power structures and ways of decisi<strong>on</strong>-making adequately<br />

into c<strong>on</strong>siderati<strong>on</strong>.<br />

R.02 The method has to give str<strong>on</strong>g support to the different maturity levels of the areas of IT<br />

Governance, IT Risk <strong>Management</strong> and IT Compliance by making them individually<br />

appraisable and improvements predictable (adjustment between AS-IS and TO-BE<br />

structures)<br />

R.03 The method requires both an initial phase of the project, as well as the<br />

establishment/transiti<strong>on</strong> into defined sustainable structures (projectoperati<strong>on</strong>al)<br />

R.04 The method must be able to rely <strong>on</strong> existing, practical c<strong>on</strong>cepts and tools (use of tools<br />

within the method)<br />

R.05 The method must support a stringent view <strong>on</strong> GRC, however, allow an appropriate use<br />

(flexibility in dealing with GRC)<br />

R.06 The method must have adequate c<strong>on</strong>diti<strong>on</strong>s for their use. The c<strong>on</strong>diti<strong>on</strong>s for the use of the<br />

method must not hamper the general discussi<strong>on</strong> of IT GRC (benefit c<strong>on</strong>tributi<strong>on</strong> must be<br />

greater than the threshold, e.g. the lack an IT strategy as a prec<strong>on</strong>diti<strong>on</strong> of IT GRC may not<br />

in general prevent the examinati<strong>on</strong> of the topic as such)<br />

R.07 The method must provide ways to take existing best practices already in use (e.g. ITIL,<br />

COBIT) adequately into account (openness of the method)<br />

4.3 Requirements validati<strong>on</strong><br />

In the (3) requirements validati<strong>on</strong> the derived requirements from the specificati<strong>on</strong> are reviewed<br />

according to existing approaches grounded in science and practice. As the research questi<strong>on</strong> and the<br />

defined requirements have a significant influence to the development of the method, this chapter has<br />

to be understood not <strong>on</strong>ly as a basic discussi<strong>on</strong>, but rather as the initial phase of the “method<br />

development” (cf. Figure 1).<br />

5. Method development<br />

Before the actual method for the adopti<strong>on</strong> of IT GRC in hospitals will be presented a c<strong>on</strong>ceptual<br />

background to ME and the meta-model of the method are given. ME as a research discipline deals<br />

with the systematic development of methods. According to Braun et al. methods are characterised<br />

through a clear goal orientati<strong>on</strong>, systematisati<strong>on</strong> and repeatability (2005). In additi<strong>on</strong>, Greiffenberg<br />

lays a special emphasis <strong>on</strong> the guidance character of methods (2003, p. 11). Gutzwiller has analysed<br />

numerous approaches to ME and has derived generally applicable method elements (1992, p. 12).<br />

According to his analysis a method is represented through five generic elements: 1) activity, 2) result,<br />

3) role, 4) technique and 4) data model. His method approach is characterised <strong>on</strong> the <strong>on</strong>e hand<br />

through an universal usability e.g., the c<strong>on</strong>scious omissi<strong>on</strong> of cardinalities between the method<br />

elements allow a certain level of latitude in terms of the purpose of the specific method elements and<br />

their arrangement and <strong>on</strong> the other hand show a wide acceptance in the IS world as it has been<br />

applied by various researches for different purposes (cf. Gericke et al. 2009; Wortmann 2006) and<br />

therefore promises a successful implementati<strong>on</strong> within the hospital envir<strong>on</strong>ment. For the research<br />

work at hand, the c<strong>on</strong>tributi<strong>on</strong> by Gutzwiller (1992) is been adapted and modified. To meet the<br />

requirements the following adjustments may have been carried out compared to the initial approach<br />

by Gutzwiller:<br />

Cardinalities have been used between the method entities in order to underline the significance<br />

and scientific rigor of the method.<br />

Additi<strong>on</strong>al method elements were added such as "phase" and process model to express the<br />

str<strong>on</strong>ger relati<strong>on</strong> of a task to a process phase (cf. Figure 3).<br />

413


Mike Krey et al.<br />

The definiti<strong>on</strong> of “hierarchy c<strong>on</strong>diti<strong>on</strong>s” is deliberately omitted. According to the understanding of<br />

the present work decomposed activities may be at the same time part of different aggregated<br />

activities (reuse), which would c<strong>on</strong>tradict a strict hierarchy.<br />

However, it seems to be useful and necessary to elaborate the meanings of the single method<br />

elements as well as their interacti<strong>on</strong> explicitly in order to derive an underlying method understanding<br />

for this research work (Table 3).<br />

Table 3: Definiti<strong>on</strong> of method elements for the adopti<strong>on</strong> of IT GRC in the hospital envir<strong>on</strong>ment<br />

Method<br />

Definiti<strong>on</strong><br />

Element<br />

Activity Task (e.g. assure support of top management) which create certain results. Task uses<br />

various techniques and can be assigned to an IT GRC area. In order to integrate<br />

existing best practices – activities of the method can be mapped to given approaches.<br />

Each task has a sequence and is hierarchically structured. Furthermore each task is<br />

Role /<br />

Stakeholder<br />

performed by a role and bel<strong>on</strong>gs to a project phase.<br />

Describes which organisati<strong>on</strong>al unit or pers<strong>on</strong> is applying the technique or activity. With<br />

regard to the hospital envir<strong>on</strong>ment the roles will be performed by members from the<br />

medical or administrative departments. As IT GRC plays a minor role in hospitals is can<br />

be assumed that e.g. a chief physician of a clinic as a stakeholder is performing a<br />

temporary role as a project member. This element refers to requirement R.01.<br />

Phase / A phase is the result of “bundled“ activities. The sequences of activities are assigned to<br />

Process Model self-c<strong>on</strong>tained phases which allow the project-driven view <strong>on</strong> each IT GRC area<br />

independent of the maturity of each area. These elements refer to requirement R.02,<br />

R.03, R05 and R.06.<br />

Result Results are recorded in previously defined and structured documents and can be taken<br />

to pieces. According to the activity “assure support of top management” the result can<br />

Technique<br />

(Formal / Soft)<br />

be a reducti<strong>on</strong> of resistance and increased project support.<br />

Techniques are assigned to formal tools or soft factors used in specific situati<strong>on</strong>s in<br />

order to deal with e.g. c<strong>on</strong>tinued and hardened situati<strong>on</strong>s that require more sensible<br />

tools. Techniques can be understood as a detailed instructi<strong>on</strong> for the development of a<br />

certain type of result. Techniques for the given activity “assure support of top<br />

management” are e.g. gain top management as project sp<strong>on</strong>sor or establish regular<br />

meetings to elicit top management requirements. The distincti<strong>on</strong> between formal and<br />

soft tools refers to requirement R.01 and R.04.<br />

IT GRC Area This element allows the hierarchical order of dedicated IT GRC areas in order to handle<br />

each aspect in more detail. E.g. in the area of IT governance further aspects of<br />

strategic alignment, value delivery, resource management or performance<br />

measurement can be supported with each activity. This element refers to requirement<br />

Best Practice<br />

Model<br />

R05 and R.06.<br />

This element refers to the requirement R.04 and R.07 and allows e.g. based <strong>on</strong> the<br />

approach by Looso (2010) the systematic adopti<strong>on</strong> of existing best practices such as<br />

COBIT.<br />

Figure 2 provides the meta-model of the method. The different method elements are modelled<br />

according to the UML 2.0 class diagram.<br />

Table 4 aims at giving the reader an understanding of the practical relevance and the final outcome of<br />

the applicati<strong>on</strong> of the meta-model. The different activities needed for the adopti<strong>on</strong> of IT GRC are<br />

preformed according to the proposed spiral model by Boehm (1986) as it combines advantages of<br />

top-down and bottom-up approaches. At the same time the iterative process model allows the hospital<br />

organisati<strong>on</strong> to become accustomed to the changed situati<strong>on</strong> e.g., through quick wins and prototyping<br />

and thereby reduces the risk of broad resistance up to a complete failure of the project. For the<br />

adopti<strong>on</strong> of IT GRC six phases have been defined. The first two phases of the process model<br />

represent an initial setting of the IT GRC topic. As the majority of the Swiss hospitals have an<br />

underdeveloped maturity of IT GRC processes the actual process model starts am<strong>on</strong>gst other things<br />

with the establishment of a comm<strong>on</strong> IT GRC understanding and a detailed stakeholder analysis<br />

including the various inhibiting and enabling power structures which might influence the adopti<strong>on</strong> of IT<br />

GRC. Phase 1 “Initiati<strong>on</strong>” aims therefore at a comm<strong>on</strong> understanding of IT GRC issues within the<br />

hospital IT department seeks for the mitigati<strong>on</strong> of project risks by c<strong>on</strong>ducting a structured stakeholder<br />

analysis and furthermore aims at the official commitment by the hospital senior management. After<br />

dealing with the topic internally, the sec<strong>on</strong>d phase (“Establishment”) aims at the establishment of IT<br />

GRC issues with dedicated favourable business partners from clinics or wards. This phase includes<br />

therefore the project marketing and lobbying with the business. After these two initial phases the<br />

414


Mike Krey et al.<br />

iterative part of the process begins. In the third phase (“Problem Understanding & Analysis”) the<br />

actual structured analysis of the different IT GRC areas and their maturities are investigated. Based<br />

<strong>on</strong> the agreed c<strong>on</strong>cept phase 4 (“Acti<strong>on</strong> Planning”) aims at the planning of c<strong>on</strong>crete acti<strong>on</strong>s in order to<br />

achieve the defined goals. The actual implementati<strong>on</strong> of the agreed acti<strong>on</strong>s is performed in phase 5<br />

(“Realisati<strong>on</strong>”). Phase 6 (“Operati<strong>on</strong>s”) turns the project result into operati<strong>on</strong>s and guarantees the<br />

c<strong>on</strong>sistency and sustainability of the achieved results by provided the resources in term of<br />

organisati<strong>on</strong>al capabilities or budget to make IT GRC an <strong>on</strong>-going and lasting approach for hospitals.<br />

However, the innermost loop is the actual staring point of the process, and thus emphasises the<br />

organisati<strong>on</strong>al, financial or even “political” c<strong>on</strong>siderati<strong>on</strong>s (uncertain power structures and resistance)<br />

which may influence the design of the first prototypes (quick-wins). Each activity performed in <strong>on</strong>e of<br />

the previous described six phases is subdivided into tasks which themselves are following a defined<br />

sequence. The roles and resp<strong>on</strong>sibilities according to each activity are documented according to a<br />

resp<strong>on</strong>sibility assignment matrix (Jacka and Keller 2009). The actual RACI matrix has been expanded<br />

by adding a “support view” which expresses the possibility that a role may assist in completing the<br />

task. The adopti<strong>on</strong> of existing best practices such as COBIT 4.1 is been guaranteed by a mapping of<br />

the IT GRC area to related COBIT 4.1 processes.<br />

Figure 2: Meta-model of the method for the adopti<strong>on</strong> of IT GRC in the hospital envir<strong>on</strong>ment<br />

6. C<strong>on</strong>clusi<strong>on</strong><br />

As ascertained by Krey et al. (2010) the self-c<strong>on</strong>tained organisati<strong>on</strong>al resp<strong>on</strong>sibility, the complexity of<br />

the processes and the lack of cross-organisati<strong>on</strong>al coordinati<strong>on</strong> make an overall comparable<br />

approach to management of assets, informati<strong>on</strong> and IT difficult. A great variety of resp<strong>on</strong>sibilities,<br />

notati<strong>on</strong>s, level of abstracti<strong>on</strong>s, tools and terminologies are the result. The present paper provides a<br />

brief introducti<strong>on</strong> into the key c<strong>on</strong>cepts of IT GRC and the current debate about the reforms within the<br />

415


Mike Krey et al.<br />

Swiss health system. It shows the drivers and benefits of both and leads to the research objective,<br />

i.e., the applicati<strong>on</strong> of IT GRC to the field of health care. The proposed meta-model is a first approach<br />

to challenges at hand. However, its evaluati<strong>on</strong> is still subject to further research (cf. Figure 1). It<br />

becomes obvious, that the selecti<strong>on</strong> of relevant validati<strong>on</strong> criteria and the use of an appropriate<br />

validati<strong>on</strong> method are crucial for the result. On the other hand the validati<strong>on</strong> framework (cf. Figure 1)<br />

should not <strong>on</strong>ly include the review of the validity of the IT GRC method but furthermore evaluate the<br />

previous identified research gap and its correctness. The development of the IT GRC method refers to<br />

an identified problem in the real world, in additi<strong>on</strong> to Cole et al. the proof of its “utility” can therefore be<br />

brought through its applicati<strong>on</strong> in the real world (2005).<br />

Table 4: Phase, activity and further method elements according to the IT GRC area of business-it<br />

alignment<br />

References<br />

Arnold, G. and Davies, M. (2000). Value-based <strong>Management</strong>: C<strong>on</strong>text and Applicati<strong>on</strong>. Wiley.<br />

Berensmann, D. (2005).Die Rolle der IT bei der Industrialisierung v<strong>on</strong> Banken. Gabler.<br />

Boehm B. (1986). A Spiral Model of Software Development and Enhancement.<br />

Braun, Chr., Wortmann, F., Hafner, M., and Winter, R. (2005). Method c<strong>on</strong>structi<strong>on</strong> - a core approach to<br />

organizati<strong>on</strong>al engineering. St. Gallen.<br />

Bruehwiler, B. (2003). Risk-<strong>Management</strong> als Fuehrungsaufgabe. Bern.<br />

Bucher, T.; Riege, Chr.; Saat, J. (2008) Evaluati<strong>on</strong> in der gestaltungsorientierten Wirtschaftsinformatik -<br />

Systematisierung nach Erkenntnisziel und Gestaltungsziel. München.<br />

Cole, R., Purao, S., Rossi, M., Sein, M. (2005). Being Proactive: Where Acti<strong>on</strong> Research Meets Design<br />

Research. Las Vegas.<br />

EDOEB, Eidgenoessischer Datenschutz- und Oeffentlichkeitsbeauftragter Leitfaden zu den technischen und<br />

organisatorischen Massnahmen des Datenschutzes<br />

http://www.edoeb.admin.ch/dokumentati<strong>on</strong>/00445/00472/00935/index.html?lang=de, accessed 2.04.2012<br />

Gericke, A; Fill, H-G.; Karagiannis, D.; Winter, R. (2009). Situati<strong>on</strong>al Method Engineering for Governance, Risk<br />

and Compliance Informati<strong>on</strong> Systems. New York.<br />

Gericke, A (2009) Problem Solving Patterns in Design Science Research - Learning from Engineering. Ver<strong>on</strong>a.<br />

Gill, S. and Purushottam, U. (2008). Integrated GRC – Is your Organizati<strong>on</strong> Ready to Move?<br />

Gutzwiller, T. (1992) Das CC RIM-Referenzmodell fuer den Entwurf v<strong>on</strong> betrieblichen, transakti<strong>on</strong>sorientierten<br />

Informati<strong>on</strong>ssystemen. Heidelberg.<br />

Greiffenberg, S. (2003). Methoden als Theorien der Wirtschaftsinformatik. Müchen.<br />

Haes, S. de and van Grembergen, W. (2008). An exploratory Study into the design of an IT Governance<br />

Minimum Baseline through Delphi Research. Communicati<strong>on</strong>s of the Associati<strong>on</strong> for Informati<strong>on</strong> Systems,<br />

22.<br />

Hevner, A. R. et al. (2004). Design Science in Informati<strong>on</strong> Systems Research. MIS Quarterly.<br />

416


Mike Krey et al.<br />

Hoerbst A, Hackl WO, Blomer R, Ammenwerth E. (2011). The status of IT service management in health care -<br />

ITIL in selected <str<strong>on</strong>g>European</str<strong>on</strong>g> countries.<br />

IT Governance Institute (2003). Board Briefing <strong>on</strong> IT Governance. IT Governance Institute.<br />

Jacka, M.; Keller, P. (2009). Business Process Mapping: Improving Customer Satisfacti<strong>on</strong>. John Wiley and S<strong>on</strong>s.<br />

Jallow, A. K. et al. (2007). Operati<strong>on</strong>al risk analysis in business processes. BT Technology Journal.<br />

Johannsen, W., Goeken, M., Just, D., and Tami, F, (2007). Referenzmodelle fuer IT-Governance: Strategische<br />

Effektivitaet und Effizienz mit COBIT, ITIL & Co. 1st ed. Heidelberg: dpunkt.<br />

Krey, M., Harriehausen, B., Knoll, M., and Furnell, S. (2010). IT Governance and its spread in Swiss Hospitals:<br />

Proceedings of the IADIS Internati<strong>on</strong>al <str<strong>on</strong>g>C<strong>on</strong>ference</str<strong>on</strong>g> e-Health 2010. p. 58 ff. Freiburg.<br />

Koebler, F., Faehling, J., Leimeister, J.M., Krcmar, H. (2010). IT Governance and Types of IT Decisi<strong>on</strong> Makers in<br />

German Hospitals.<br />

Looso, St. (2010). Towards a structured applicati<strong>on</strong> of IT governance best practice reference models.<br />

March, S. T., and Smith, G. F. (1995). Design and natural science research <strong>on</strong> informati<strong>on</strong> technology. Decisi<strong>on</strong><br />

Support Systems 15.<br />

Mirbel, I. and Ralyté, J. (2006). Situati<strong>on</strong>al method engineering: combining assembly-based and roadmap-driven<br />

approaches. Requirements Engineering.<br />

OECD and WHO (2011). OECD Reviews of Health Systems: OECD Publishing.<br />

Piccoli, G., and Ives, B. (2005). IT-dependent Strategic Initiatives and Sustained Competitive Advantage: A<br />

Review and Synthesis of the Literature. MIS Quarterly.<br />

Porter, M. E., Teisberg, E. O (2004). Redefining competiti<strong>on</strong> in health care. Bost<strong>on</strong>, MA.<br />

Proctor, P. E. (2005) Select and Implement Appropriate C<strong>on</strong>trols for Regulatory Compliance.<br />

PWC (2006): Governance, Risk and Compliance, An Integrated Approach for the Insurance Industry. Washingt<strong>on</strong><br />

D.C.<br />

Rausch, T. (2006). Holistic business process and compliance management. In: Pour, J. and Vorisek, J. (Eds.):<br />

14th Internati<strong>on</strong>al <str<strong>on</strong>g>C<strong>on</strong>ference</str<strong>on</strong>g> Systems Integrati<strong>on</strong> (SI 2006), Prague.<br />

Rockwell Schulz, Alt<strong>on</strong> C. (2005). Johns<strong>on</strong> <strong>Management</strong> of Hospitals and Health Services: Strategic Issues and<br />

Performance.<br />

Sommerville, I. (2011) Software Engineering, 9/E, Addis<strong>on</strong>-Wesley.<br />

Steinberg, R. M. et al. (2004). Enterprise Risk <strong>Management</strong> Framework.<br />

U.S. Department of Health & Human Services (2003) Health Insurance Reform: Security Standards,"HHS,<br />

Washingt<strong>on</strong>, 2003 http://aspe.hhs.gov/admnsimp/final/fr03-8334.pdf, accessed, 21.04.2012<br />

VIG (2005), Verein fuer Informatik im Gesundheitswesen (VIG). E-Health-Strategie fuer die Instituti<strong>on</strong>en im<br />

Gesundheitswesen des Kant<strong>on</strong>s St. Gallen: Grundlagenpapier. accessed February 2012.<br />

Weill, P.; Ross, J. (2004). IT-Governance. How top performers manage IT decisi<strong>on</strong> rights for superior results.<br />

Bost<strong>on</strong>, USA: Harvard Business School Press.<br />

Wortmann, F. (2006). Entwicklung einer Methode fuer die unternehmensweite Autorisierung. Universitaet<br />

St.Gallen, St.Gallen.<br />

417


Collaborative Methodology for Supply Chain Quality<br />

<strong>Management</strong>: Framework and Integrati<strong>on</strong> With Strategic<br />

Decisi<strong>on</strong> Processes in Product Development<br />

Juan Camilo Romero 1,2 , Thierry Coudert 1 , Laurent Geneste 1 and Aymeric De<br />

Valroger 2<br />

1<br />

Laboratoire Génie de Producti<strong>on</strong> / INPT-ENIT / University of Toulouse, Tarbes,<br />

France<br />

2<br />

AXSENS SAS, Toulouse, France<br />

juan.romero@axsens.com<br />

thierry.coudert@enit.fr<br />

laurent.geneste@enit.fr<br />

aymeric.devalroger@axsens.com<br />

Abstract: The new generati<strong>on</strong> of network-based organizati<strong>on</strong>s has triggered the emergence of distributed and<br />

more complex c<strong>on</strong>texts for the analysis of firms’ strategies. This gradual change in the way we understand<br />

enterprises has induced radical evoluti<strong>on</strong>s <strong>on</strong> the Quality <strong>Management</strong> domain. As a c<strong>on</strong>sequence, the Problem<br />

Solving Methodologies (PSM) widely used in industry and positi<strong>on</strong>ed up to now as <strong>on</strong>e of the key elements for<br />

achieving c<strong>on</strong>tinuous improvement efforts within local scopes are now insufficient to deal with major and<br />

distributed problems and requirements in this new envir<strong>on</strong>ment. The definiti<strong>on</strong> of a generic and collaborative<br />

PSM well-adapted to supply chain c<strong>on</strong>texts is <strong>on</strong>e of the purposes of this paper. Additi<strong>on</strong>al requirements linked to<br />

specificities carried out by the introducti<strong>on</strong> of a networked c<strong>on</strong>text within the methodology scope, the relati<strong>on</strong>al<br />

aspects of the supply chains, complexity and distributi<strong>on</strong> of informati<strong>on</strong>, distributed decisi<strong>on</strong>-making processes<br />

and knowledge management challenges are some of the aspects being addressed by the proposed<br />

methodology. A special focus is made <strong>on</strong> benefits obtained through the integrati<strong>on</strong> of those elements across all<br />

problem-solving phases and particularly a proposal for multi-level root-cause analysis articulating both horiz<strong>on</strong>tal<br />

and vertical decisi<strong>on</strong> processes of supply chains is presented. In additi<strong>on</strong> to laying out the expected benefits of<br />

such a methodology in the Quality <strong>Management</strong> area, the article studies the reuse of all the quality-related<br />

evidence capitalized in series phase as a driver for improving upstream phases of product development projects.<br />

This paper addresses this link between series and development activities in light of the proposed PSM and<br />

intends to encourage discussi<strong>on</strong> <strong>on</strong> the definiti<strong>on</strong> of new approaches for Quality <strong>Management</strong> throughout the<br />

whole product lifecycle. Some enabling elements in the decisi<strong>on</strong>-making processes linked to both the problemsolving<br />

in series phase and the roll-out of new products are introduced.<br />

Keywords: problem solving methodology, supply chain quality management, product development, experience<br />

feedback, collaborative supply chains<br />

1. Introducti<strong>on</strong><br />

In the past, firms used to work based essentially <strong>on</strong> achievement of local objectives. This way of<br />

working produced short-sighted, standal<strong>on</strong>e and c<strong>on</strong>flicting strategies between firms and their<br />

stakeholders, which led up to misalignment and poor global performances. In that c<strong>on</strong>text, the quality<br />

was c<strong>on</strong>sequently managed in a reduced perimeter characterized by local c<strong>on</strong>tinuous improvement<br />

efforts (Foster 2007). For instance, centralized PSM well-adapted for dealing with local problems<br />

gained a place as a cornerst<strong>on</strong>e element within firms’ strategies for meeting quality exigencies.<br />

N<strong>on</strong>etheless, the higher levels of competiti<strong>on</strong> and the intensificati<strong>on</strong> of cost, quality and delivery<br />

requirements have forced enterprises to cross their own boundaries towards more collaborative<br />

models involving stakeholders (Derrouiche 2008). Thereby, approaches based <strong>on</strong> the noti<strong>on</strong> of an<br />

Extended Enterprise and including principles such as objectives alignment, strategy synchr<strong>on</strong>izati<strong>on</strong>,<br />

collaborative practices and comm<strong>on</strong>-to-all processes appeared (Cao et al. 2011). This new<br />

envir<strong>on</strong>ment led up to the emergence of Collaborative Supply Chains defined as self-organized<br />

networks of organizati<strong>on</strong>s acting as a whole and looking for global fulfillment of final customer needs<br />

(Knowles et al. 2005).<br />

The introducti<strong>on</strong> of this c<strong>on</strong>cept has radically changed the quality management paradigms. The<br />

c<strong>on</strong>tinuous improvement practices, such as the PSM, are now faced to deal with networked,<br />

distributed and more complex envir<strong>on</strong>ments including am<strong>on</strong>gst others a larger number of partners,<br />

huge quantities of fragmented and distributed informati<strong>on</strong>, higher impact problems defined at a supply<br />

418


Juan Camilo Romero et al.<br />

chain level, distributed and no more centralized activities and collaborative aspects not addressed<br />

before.<br />

The aim of this paper is to meet the challenge of successfully extending the PSM and their benefits to<br />

Supply Chain c<strong>on</strong>texts and intends to propose a generic and collaborative methodology adapted to<br />

new-generati<strong>on</strong> of network-based organizati<strong>on</strong>s. The proposed methodology aims to be a driver for<br />

the enhancement of quality management and c<strong>on</strong>tinuous improvement efforts at a supply chain level.<br />

It has been improved with a two-layered structure for modeling all the technical and collaborative<br />

aspects of supply chains and synchr<strong>on</strong>ized with a distributed Experience Feedback System enabling<br />

the methodology dealing with knowledge management across distributed c<strong>on</strong>texts. This methodology<br />

is presented in Secti<strong>on</strong> 2.<br />

Once the proposed global methodology has been deployed across all the stages of the Supply Chain,<br />

access to huge quantity of meaningful and structured quality-related informati<strong>on</strong> capitalized for<br />

products and partners in series phase is available to be exploited. All the problem solving experiences<br />

can thus be reused not <strong>on</strong>ly to enhance solving of new problems in series phase but also as an input<br />

for improving and enhancing new products design in development <strong>on</strong>es. In Secti<strong>on</strong> 3, the general<br />

guidelines for a global c<strong>on</strong>tinuous improvement approach linking series and development phases are<br />

presented.<br />

2. A collaborative problem solving methodology adapted to supply chain<br />

c<strong>on</strong>texts<br />

In order to deal with all the critical aspects of new generati<strong>on</strong> of network-based organizati<strong>on</strong>s, the<br />

proposed PSM has been defined as a whole soluti<strong>on</strong> composed by three cornerst<strong>on</strong>e elements as<br />

shown in Figure 1. The first element is defined by the Methodology itself and aims to structure the<br />

whole solving process. The sec<strong>on</strong>d element is based <strong>on</strong> a Two-layered Model dealing with the study<br />

of distributed c<strong>on</strong>texts. This model is composed of two complementary levels handling with both<br />

technical and collaborati<strong>on</strong> dimensi<strong>on</strong>s of supply chains. The third pillar of the methodology<br />

corresp<strong>on</strong>ds to the distributed Experience Feedback Process dealing with the capitalizati<strong>on</strong> and reuse<br />

of problem solving experiences through networked and distributed c<strong>on</strong>texts.<br />

Figure 1: The global problem solving methodology<br />

The Methodology c<strong>on</strong>siders that the Problem Solving is a generic process that can be understood<br />

from a simplified approach with four phases: C<strong>on</strong>text, Analysis, Soluti<strong>on</strong> and Less<strong>on</strong>-Learnt (Kamsu-<br />

Foguem et al. 2008). The fact that existing methodologies for problem solving such as the plan-docheck-act<br />

(PDCA), the 8-Disciplines (8D) and the six-sigma DMAIC (Define, Measure, Analyze,<br />

Improve and C<strong>on</strong>trol) can be expressed in terms of these four standard phases provides this choice<br />

419


Juan Camilo Romero et al.<br />

with a generic reas<strong>on</strong>ing c<strong>on</strong>tributing to adaptability and deployment in a wide range of industrial<br />

c<strong>on</strong>texts (Jabrouni et al. 2010).<br />

2.1 Specificati<strong>on</strong> of the problem c<strong>on</strong>text<br />

The C<strong>on</strong>text phase of the Methodology aims to border the problem by keeping <strong>on</strong>ly relevant<br />

informati<strong>on</strong> c<strong>on</strong>tributing to problem understanding and providing with meaningful evidence for further<br />

solving phases. Due to the nature of problems to be handled within the frame of network-based<br />

organizati<strong>on</strong>s, the quantity and quality of informati<strong>on</strong> become a critical issue (Cantor et al. 2009). The<br />

proposed methodology specifies a relevant problem c<strong>on</strong>text through the accomplishment of four<br />

phases: Formalizati<strong>on</strong>, Filtering, Pilot Team c<strong>on</strong>stituti<strong>on</strong> and Reusing.<br />

2.1.1 Formalizati<strong>on</strong><br />

When dealing with complex c<strong>on</strong>texts characterized by the multiplicity, fragmentati<strong>on</strong>, distributi<strong>on</strong> and<br />

heterogeneity of informati<strong>on</strong> across a network, the first step is to define comm<strong>on</strong>-to-all frameworks<br />

and harm<strong>on</strong>ized mechanisms for formalizati<strong>on</strong> and sharing of that informati<strong>on</strong> (Buz<strong>on</strong> et al. 2007)<br />

(Zhou et al. 2007). This first objective has been achieved by coupling the methodology with a<br />

proposal for the modeling, diagnostic and analysis of supply chains in the light of problem solving<br />

processes. This model corresp<strong>on</strong>ds to the sec<strong>on</strong>d cornerst<strong>on</strong>e element of the global methodology as<br />

shown in the Figure 1.<br />

Such approach is defined by two complementary levels addressing the whole technical and<br />

collaborati<strong>on</strong> dimensi<strong>on</strong>s of Supply Chains through a unique two-layered model. The first technical<br />

layer aims to address the entire product, process and network related aspects while the sec<strong>on</strong>d <strong>on</strong>e<br />

intends to complete the model by dealing with all the relati<strong>on</strong>al and collaborati<strong>on</strong> aspects of supply<br />

chains. Each level is materialized in the proposed model through a breakdown structure modulating<br />

both technical and collaborati<strong>on</strong> aspects in more manageable units. Then, the Technical Breakdown<br />

Structure (TBS) or first layer is thus defined by the set of all the Technical Packages (TP)<br />

summarizing all the technical attributes for each node of the network while the Collaborati<strong>on</strong><br />

Breakdown Structure (CBS) or sec<strong>on</strong>d layer is defined by the set of all the Collaborati<strong>on</strong> Packages<br />

(CP) representing the clustering of partners across the same network for solving purposes. Unlike the<br />

first level, characterized as static and aiming to provide robustness to the model, the sec<strong>on</strong>d <strong>on</strong>e is<br />

defined dynamically with regard to supply chain collaborati<strong>on</strong> aspects evolving c<strong>on</strong>stantly through the<br />

time and being specific for each problem. Aspects such as c<strong>on</strong>fidentiality, trust and power between<br />

partners corresp<strong>on</strong>d to criteria being critical <strong>on</strong> the effectiveness of Supply Chain practices operati<strong>on</strong>.<br />

Other characteristic of this sec<strong>on</strong>d level is that it can be modulated through multiple sub-levels<br />

representing nested and dynamic collaborati<strong>on</strong> structures. This adaptability and flexibility allow the<br />

model to provide up-to-date and reliable c<strong>on</strong>texts in the light of problem solving processes. Figure 2<br />

synthesizes and positi<strong>on</strong>s the different aspects of the TBS+CBS model.<br />

Figure 2: The two-layered approach for the modeling of supply chains<br />

420


Juan Camilo Romero et al.<br />

The Formalizati<strong>on</strong> phase, enabling the specificati<strong>on</strong> of relevant c<strong>on</strong>texts within networked<br />

envir<strong>on</strong>ments, is based <strong>on</strong>: (1) the first level of the two-layered model summarizing the technical<br />

c<strong>on</strong>text of the supply chain where problems are occurring and (2) the entire Collaborati<strong>on</strong> Knowledge<br />

generalized from past problem solving experiences. At this stage, available Technical Breakdown<br />

Structure (TBS) and Collaborati<strong>on</strong> Knowledge (CK) provided by the model are defined at a global<br />

Supply Chain level and do not include yet all the specificities linked to current problems being solved.<br />

2.1.2 Filtering<br />

This phase focuses <strong>on</strong> the filtering of the networked c<strong>on</strong>texts issued from previous steps with regards<br />

to specific problems in order to keep <strong>on</strong>ly the relevant problem-related informati<strong>on</strong>. To do so, a<br />

mechanism intending to filter the c<strong>on</strong>text issued from the two-layered model and get thus a simplified<br />

TBS adapted to current problem has been specified. This Filtering Mechanism (FM), aiming to keep<br />

the most relevant TPs for the current problem solving experience, is based <strong>on</strong> the assumpti<strong>on</strong> that<br />

when a problem appears, the partner directly c<strong>on</strong>cerned by this problem is able to define a reduced<br />

number of relevant problem-related criteria. These criteria are the inputs for the filtering mechanism<br />

and are formalized through a Preliminary Problem C<strong>on</strong>text Record (PPCR) including:<br />

Impacted element: Product <strong>on</strong> which the problem has been detected. After it has been identified,<br />

the corresp<strong>on</strong>ding TP summarizing all the product, process and network dimensi<strong>on</strong>s for this<br />

element can be identified.<br />

Problem width: Criteri<strong>on</strong> based <strong>on</strong> the selecti<strong>on</strong> of the more relevant n-1 level TPs. It allows<br />

reducing the number of branches (or width) of the TBS to be analyzed.<br />

Problem depth: Criteri<strong>on</strong> defined in functi<strong>on</strong> of the number of levels to be included at the analysis<br />

and filtering stages. It allows reducing the depth of the TBS.<br />

Problem domain: Criteri<strong>on</strong> allowing characterizing current problem with regard to pre-defined<br />

types of problems. The domains characterizing current problem are then matched with the<br />

domains characterizing the TPs of the TBS in order to reduce the already filtered scope.<br />

Relevant processes: Criteri<strong>on</strong> defined in functi<strong>on</strong> of processes being c<strong>on</strong>sidered as relevant for<br />

current problem solving experience. This choice is d<strong>on</strong>e taking into account the Design,<br />

Industrializati<strong>on</strong>, Fabricati<strong>on</strong> and Transport dimensi<strong>on</strong>s studied by the TPs. It allows reducing for<br />

already filtered TPs, the quantity and nature of informati<strong>on</strong> to be kept.<br />

The filtering process computes and matches all these problem-related criteria with the attributes<br />

characterizing the TPs across the whole TBS. The definiti<strong>on</strong> of these criteria is initialized based <strong>on</strong><br />

preliminary analysis performed with regard to problem evidence available at early stages of the<br />

solving process. Nevertheless, it is completed throughout the methodology in order to improve filtering<br />

results. The output of this phase is a simplified TBS adapted to current problem (identified as TBS’).<br />

2.1.3 Pilot team c<strong>on</strong>stituti<strong>on</strong><br />

This phase aims to identify the actors that will pilot the solving efforts. Based <strong>on</strong> the previously<br />

simplified TBS’ and taking into account that partners are linked to TPs, the model is able to identify all<br />

the relevant actors owning key processes and having problem-related competencies. This set of<br />

eligible c<strong>on</strong>tributors, grouping the partners being well-positi<strong>on</strong>ed across the network to potentially<br />

c<strong>on</strong>tribute to problem solving, can be updated manually throughout the methodology by adding actors<br />

in functi<strong>on</strong> of specific-to-problem requirements (e.g. domain experts or authority representatives). The<br />

selecti<strong>on</strong> of the pilot team members, issued from this set of eligible c<strong>on</strong>tributors, is executed by the<br />

partner directly c<strong>on</strong>cerned by the problem through a Collaborati<strong>on</strong> Mechanism (CM). This<br />

mechanism, aiming to optimize the pilot team c<strong>on</strong>stituti<strong>on</strong> in regards to additi<strong>on</strong>al behavioral and<br />

collaborative aspects of the supply chains, led up to the definiti<strong>on</strong> of the sec<strong>on</strong>d level of the two<br />

layered model. The inputs for this mechanism are based <strong>on</strong> the <strong>on</strong>e side <strong>on</strong> the Collaborati<strong>on</strong><br />

Knowledge generalized for the c<strong>on</strong>cerned Supply Chain and <strong>on</strong> the other <strong>on</strong>e <strong>on</strong> the Partner<br />

Preferences Record (PPR) summarizing partners’ preferences in regards to critical supply chain<br />

collaborati<strong>on</strong> criteria such as power, trust, c<strong>on</strong>trol, objectives alignment, informati<strong>on</strong> sharing and<br />

c<strong>on</strong>flict (Cao et al. 2011). The output of this phase is the pilot team which is positi<strong>on</strong>ed as the header<br />

element of the sec<strong>on</strong>d layer (CBS0) of the two-layered model (TBS+CBS).<br />

421


Juan Camilo Romero et al.<br />

2.1.4 Reusing<br />

This step intends to exploit the shared knowledge database. The purpose is to find the most relevant<br />

past problem solving experiences in order to enhance the current <strong>on</strong>e. This phase is executed by the<br />

pilot team and is based <strong>on</strong> the completi<strong>on</strong> of the previously defined problem-related criteria, this time<br />

through a C<strong>on</strong>solidated Problem C<strong>on</strong>text Record (CPCR). This record is used as the input for the<br />

Reusing Mechanism (RM) whose output is the set of all the relevant past solving experiences having<br />

the higher similarity degree and providing with meaningful informati<strong>on</strong> being able to be<br />

adapted/reused in the current solving experience.<br />

The output of the C<strong>on</strong>text phase corresp<strong>on</strong>ds to a problem c<strong>on</strong>text characterized by the<br />

completeness, relevance, readiness and accuracy of its c<strong>on</strong>tent. This reduced but relevant c<strong>on</strong>text<br />

provides problem solvers with meaningful and added-value informati<strong>on</strong> that enhances the decisi<strong>on</strong>making<br />

processes all al<strong>on</strong>g the methodology. The possibility of reducing from a huge space of<br />

research to a more reduced scope with prior and more relevant problem-related informati<strong>on</strong> and the<br />

opportunity of reusing and adapting past problem solving experiences are fundamental gains that can<br />

be reached through the deployment of the proposed methodology. All the key elements of the C<strong>on</strong>text<br />

phase are synthetized in Figure 3.<br />

Figure 3: Key elements of the c<strong>on</strong>text phase<br />

2.2 Multi-level root-cause analysis<br />

The Analysis is performed <strong>on</strong> the basis of the c<strong>on</strong>text issued from the previous phase. The objective<br />

is to perform a deep analysis of the problem and available informati<strong>on</strong> in order to find the root causes<br />

producing the problem. The identificati<strong>on</strong> of those causes is a critical factor within the quality domain<br />

as their identificati<strong>on</strong> is a mean for allowing a full eradicati<strong>on</strong> of problems (Wils<strong>on</strong> et al. 1993). The<br />

problem analysis efforts within the frame of supply chains demands from the methodology additi<strong>on</strong>al<br />

assets such as: (1) synchr<strong>on</strong>izati<strong>on</strong> of distributed partners in regards of comm<strong>on</strong> decisi<strong>on</strong> processes,<br />

(2) clustering in order to optimize competencies, promote synergies and mitigate risks, (3)<br />

establishment of shared processes in order to improve decisi<strong>on</strong> flows and (4) dealing with relati<strong>on</strong>al<br />

and collaborati<strong>on</strong> aspects of the supply chains in order to create positive communicati<strong>on</strong> and<br />

establishment of intensive l<strong>on</strong>g-term relati<strong>on</strong>ships. Within this c<strong>on</strong>text and as part of the current<br />

methodology, a proposal dealing with a dynamic and multi-level root-cause analysis adapted to<br />

Supply Chain problems has been specified.<br />

This proposal is based <strong>on</strong> the coupling of two elements: the causal tree of a problem and the twolayered<br />

model. The first element is widely used within the c<strong>on</strong>tinuous improvement area and the<br />

sec<strong>on</strong>d <strong>on</strong>e aims to enhance distributed decisi<strong>on</strong> processes by incorporating the whole technical and<br />

collaborative dimensi<strong>on</strong>s related to Supply Chains. The causal tree, breaking down problems until the<br />

root-causes are found, is c<strong>on</strong>sidered as the driver of the proposed approach while the two-layered<br />

422


Juan Camilo Romero et al.<br />

model is c<strong>on</strong>sidered as the facilitator of this proposal within distributed c<strong>on</strong>texts. The inputs for the<br />

analysis phase are the problem c<strong>on</strong>text, the pilot team and the set of all the eligible c<strong>on</strong>tributors<br />

issued from previous phase. These elements are materialized in the two-layered model by the TBS’<br />

and the header level of the CBS c<strong>on</strong>taining the pilot team.<br />

For each level of the causal tree, a corresp<strong>on</strong>ding collaborative structure being able to deal with the<br />

analysis of elements included at that level must be defined. Each collaborative structure,<br />

corresp<strong>on</strong>ding to <strong>on</strong>e level of the CBS and being defined by a set of interc<strong>on</strong>nected CPs, must fulfill<br />

the whole Supply Chains requirements. As a c<strong>on</strong>sequence, during the clustering of partners with<br />

problem analysis purposes not <strong>on</strong>ly technical issues and competencies owned by actors but also all<br />

the relati<strong>on</strong>al and collaborati<strong>on</strong> aspects must be c<strong>on</strong>sidered. For the header level of the causal tree<br />

dealing with the problem (H0), this requirement has been fulfilled through the applicati<strong>on</strong> of the<br />

Collaborati<strong>on</strong> Mechanism (CM) enabling the c<strong>on</strong>stituti<strong>on</strong> of the pilot team. This team, as shown in<br />

Figure 4, is positi<strong>on</strong>ed at the header level of the CBS (CBS0) and has the accountability of analyzing<br />

the corresp<strong>on</strong>ding header level of the causal tree (H0).<br />

Once the pilot team has analyzed the problem level in the light of available technical c<strong>on</strong>text issued<br />

from the TBS’, it must breakdown the problem into more manageable level-1 causes (H1) potentially<br />

producing this problem. These are the causes answering to the questi<strong>on</strong>: Why has the current-level<br />

cause (H0) occurred? When level-1 causes (H1) are identified, the first level of the CBS (CBS1) is<br />

initialized. This level includes <strong>on</strong>e Collaborati<strong>on</strong> Package (CP) for each cause <strong>on</strong> the corresp<strong>on</strong>ding<br />

level of the causal tree. For the definiti<strong>on</strong> of the team members bel<strong>on</strong>ging to each CP, the same<br />

Collaborati<strong>on</strong> Mechanism (CM) used for the c<strong>on</strong>stituti<strong>on</strong> of the pilot team is deployed. This<br />

mechanism, computing all the Partner Preferences Record (PPR) for partners included in the eligible<br />

c<strong>on</strong>tributors set, optimize the resources distributi<strong>on</strong> and the clustering of partners in regards of the<br />

analysis and validati<strong>on</strong> of current problem causes and behavioral aspects. After each CP is defined,<br />

the accountabilities and the different roles inside the package are distributed with regard to agreed<br />

Collaborati<strong>on</strong> Agreements governing CP operati<strong>on</strong>. To favor communicati<strong>on</strong> between different levels<br />

of collaborative structures, the CP coordinator at <strong>on</strong>e level must be at least a CP member at the n-1<br />

level. At this stage, the CPs are ready to start the analysis of the level-1 causes (H1) in the light of<br />

available technical c<strong>on</strong>text issued from the TBS’. From this point, all steps are reproduced within the<br />

frame of a recursive approach intending to breakdown in a synchr<strong>on</strong>ized way the problem and the<br />

collaborative structures until the root-causes (n-level) are finally found. The principles of this<br />

collaborative approach positi<strong>on</strong>ed as a driver for complex problems analysis <strong>on</strong> extended and<br />

distributed c<strong>on</strong>texts and its integrati<strong>on</strong> with the two-layered model (TBS + CBS) are illustrated <strong>on</strong><br />

Figure 4.<br />

A top-down and a bottom-up flow covering the causal tree and the definiti<strong>on</strong> of dynamic teams or<br />

Collaborati<strong>on</strong> Packages (CP) across the CBS in a recursive way are the backb<strong>on</strong>e elements of the<br />

global decisi<strong>on</strong> process of this model. The top-down flow aims to analyze and distribute causes<br />

through the CBS levels while the bottom-up flow aims to validate analysis and provide c<strong>on</strong>solidated<br />

results. In both descending and ascending flows team collaborative work is deployed in order to align<br />

and coordinate efforts. After both flows have been completed, a c<strong>on</strong>solidated causal tree listing all<br />

relevant root-causes producing the current problem is capitalized into the global knowledge database.<br />

Synchr<strong>on</strong>izati<strong>on</strong> of Supply Chain and local decisi<strong>on</strong> processes are included as a way to promote the<br />

enhancement of Learning Organizati<strong>on</strong>s. Through this approach the resources participating to the<br />

problem analysis process are optimized and both individual and network competencies and<br />

knowledge are c<strong>on</strong>solidated.<br />

From a more global point of view, the possibility of having a central repository c<strong>on</strong>taining all the rootcauses<br />

producing problems being detected <strong>on</strong> products moving through supply chains and the<br />

corresp<strong>on</strong>ding collaborative structures performing these analyses provides a very important<br />

competitive advantage in regards of strategic decisi<strong>on</strong> processes at a supply chain level. All this<br />

quality-related informati<strong>on</strong> can c<strong>on</strong>sequently be exploited for enhancing the Supply Chain Quality<br />

<strong>Management</strong> (SCQM) through the definiti<strong>on</strong> of generalized c<strong>on</strong>tinuous improvement efforts tackling<br />

and eradicating recurrent and high impact Supply Chain problems.<br />

423


Juan Camilo Romero et al.<br />

Figure 4: A multi-level approach for root-cause analysis <strong>on</strong> distributed c<strong>on</strong>texts<br />

2.3 Soluti<strong>on</strong> and less<strong>on</strong> learnt phases<br />

In this phase, the two-layered model allows focusing the team collaborative efforts <strong>on</strong> the definiti<strong>on</strong> of<br />

an acti<strong>on</strong> plan addressing the root causes. The same top-down and bottom-up flows can be used now<br />

to deploy an acti<strong>on</strong> plan distributed horiz<strong>on</strong>tally through the different stages of the network and<br />

vertically through the different organizati<strong>on</strong> decisi<strong>on</strong> levels. A global and aggregated approach<br />

synchr<strong>on</strong>izing vertical and horiz<strong>on</strong>tal flows of supply chain ensures the effectiveness of the soluti<strong>on</strong>s<br />

deployed and c<strong>on</strong>tributes to the achieving of global c<strong>on</strong>tinuous improvement objectives defined within<br />

the frame of the SCQM efforts (Harland et al. 2004).<br />

The Less<strong>on</strong> Learnt phase intends to encapsulate the whole PSM, related activities and associated<br />

knowledge in <strong>on</strong>e individual experience being capitalized into the shared knowledge database. At this<br />

stage, great knowledge management benefits can be obtained because both global and local qualityoriented<br />

competencies are created, shared and distributed across the network. This allows gaining <strong>on</strong><br />

higher performances and obtaining superior Supply Chain competitiveness levels.<br />

3. The proposed methodology as a driver for enhancing strategic making<br />

decisi<strong>on</strong> processes during new product development<br />

The new product development projects include some strategic activities having important impacts<br />

over the whole product lifecycle. Two of these crucial activities c<strong>on</strong>cern <strong>on</strong> the <strong>on</strong>e hand the system<br />

design process aiming to define and freeze product plans through both preliminary and detailed<br />

design reviews (Mavris et al 2011) and <strong>on</strong> the other <strong>on</strong>e the supplier selecti<strong>on</strong> phase aiming to<br />

identify, evaluate, and c<strong>on</strong>tract with suppliers (Beil 2010). These two phases, being part of the<br />

classical product development approaches, have been retained for the purpose of this article as they<br />

allow highlighting some c<strong>on</strong>crete links existing between the proposed global methodology for quality<br />

management in series phases and the new products roll-out in the development <strong>on</strong>e (see Figure 5).<br />

3.1 System design process<br />

The system design process encompasses the process during which a new product is brought from<br />

the c<strong>on</strong>ceptual stage to readiness for full-scale producti<strong>on</strong>. All al<strong>on</strong>g this process including some<br />

preliminary, detailed and critical reviews, the system structure evolves through different maturity<br />

stages with different kind of business, technical, industrial, quality and risk factors being leveraged<br />

424


Juan Camilo Romero et al.<br />

(Handfield 1999). In additi<strong>on</strong>, sometimes the less<strong>on</strong>s learnt from previous development projects are<br />

also included throughout these stages to improve the current system specificati<strong>on</strong> (Vareilles et al.<br />

2012). N<strong>on</strong>etheless, the informati<strong>on</strong> provided by past development experiences and by classical<br />

product development approaches can be completed and enhanced through the applicati<strong>on</strong> of<br />

structured knowledge processes defined in a larger scope including the whole product lifecycle. For<br />

instance, when the system is being specified, it could be useful to have access to all the qualityrelated<br />

informati<strong>on</strong> capitalized for similar and/or same family systems in series phase through the<br />

proposed PSM.<br />

Figure 5: Integrati<strong>on</strong> of the PSM into strategic decisi<strong>on</strong>-making in development phases<br />

As detailed <strong>on</strong> Secti<strong>on</strong> 2, after the global methodology has been deployed across all stages of the<br />

supply chain, a central knowledge base including meaningful informati<strong>on</strong> in regards of new product<br />

development is available to be exploited. List of some structured, added-value and meaningful<br />

informati<strong>on</strong> available from this central quality repository includes but is not limited to:<br />

All problems detected for c<strong>on</strong>stituents, similar and/or same-family systems,<br />

Relevant root-causes analysis used for solving those problems,<br />

Soluti<strong>on</strong>s proposed and related acti<strong>on</strong> plans,<br />

Generalized knowledge issued from the less<strong>on</strong> learnt phases and,<br />

Relevant technical c<strong>on</strong>text (TBS’) and collaborati<strong>on</strong> structures (CBS) deployed.<br />

All this global relevant informati<strong>on</strong> encapsulated in the problem solving experiences can be exploited<br />

through the applicati<strong>on</strong> of the Reusing Mechanism (RM) defined in Secti<strong>on</strong> 2.1.4. A Design<br />

Requirements Record (DRR), characterizing the whole criteria specific to design/development<br />

activities, enables the reusing/adaptati<strong>on</strong> process. This last <strong>on</strong>e aims to reuse and keep <strong>on</strong>ly the more<br />

relevant informati<strong>on</strong> in order to: (1) highlight risks not before c<strong>on</strong>sidered for the current system<br />

development, (2) improve current design by leveraging all problems occurred <strong>on</strong> similar or same<br />

family systems/comp<strong>on</strong>ents, (3) justify functi<strong>on</strong>al and structural choices for materials and/or<br />

comp<strong>on</strong>ents in the light of proved performances, (4) find design alternatives for evaluati<strong>on</strong> of<br />

ec<strong>on</strong>omic scenarios and finally (5) boost the supplier selecti<strong>on</strong> phase.<br />

3.2 Supplier selecti<strong>on</strong><br />

It could be useful to access during the supplier qualificati<strong>on</strong>/selecti<strong>on</strong> process to relevant informati<strong>on</strong><br />

measuring partner involvement <strong>on</strong> collaborative practices already deployed across the whole supply<br />

chain. The proposed approach analyzes relevant informati<strong>on</strong> capitalized through the global PSM in<br />

425


Juan Camilo Romero et al.<br />

order to come up with two complementary indicators aiming to measure this involvement and enabling<br />

decisi<strong>on</strong>-making at this early stage of suppliers qualificati<strong>on</strong>/selecti<strong>on</strong>:<br />

A first collaborativity index intends to define the degree of involvement, adherence and alignment<br />

of partners with quality strategies of collaborative supply chains. So, more the firms work in<br />

partnership with stakeholders to solve supply chain problems and more they deploy the proposed<br />

PSM, more its collaborativity index improves. This index, measuring the disposal degree of a<br />

partner to work in a collaborative way, can be reas<strong>on</strong>ed from the past problem solving<br />

experiences including am<strong>on</strong>gst others the entire CBS and CK c<strong>on</strong>text.<br />

A sec<strong>on</strong>d risk-oriented index defined for each supplier/product couple can be aggregated from<br />

informati<strong>on</strong> available in the central knowledge repository. The number of high-impact problems<br />

detected for this couple and the related analyses performed can represent meaningful evidence<br />

during supplier assessment. This measure can be aggregated in order to define a supplier risk<br />

index aiming to reflect the trustworthiness associated to partners and their already industrialized<br />

products. The risk-oriented index must be analyzed in parallel with the collaborativity in order to<br />

favor the involvement of partners. The fact of increasingly cooperating with the other partners to<br />

solve problems will allow firms c<strong>on</strong>solidating competencies and enhancing local processes, which<br />

c<strong>on</strong>sist in the l<strong>on</strong>g-term <strong>on</strong> a driver for improving product quality and reducing risks. Then, the<br />

involvement and engagement of partners with the proposed methodology represents a double<br />

gain.<br />

Proposed measures enhance supplier qualificati<strong>on</strong> process by providing with additi<strong>on</strong>al meaningful<br />

informati<strong>on</strong> not being before deeply analyzed due to: (1) complexity of gathering this kind of<br />

informati<strong>on</strong> <strong>on</strong> distributed and more complex c<strong>on</strong>texts such as the supply chains and (2) the lack of<br />

maturity of links existing between series and development phases in the quality management area.<br />

These new indicators must be understood as two complementary and interdependent elements for<br />

assessing suppliers’ performance <strong>on</strong> problem solving and c<strong>on</strong>tinuous improvement processes.<br />

Finally, this set of index can be articulated by the buyers with more global sourcing strategies in order<br />

to promote higher involvement from suppliers in supply chain collaborative practices.<br />

4. C<strong>on</strong>clusi<strong>on</strong><br />

In this paper, a collaborative methodology for problem solving adapted to supply chain c<strong>on</strong>texts has<br />

been defined. It has been positi<strong>on</strong>ed as a key driver for achieving an effective Supply Chain Quality<br />

<strong>Management</strong>. The methodology addresses the whole technical and collaborati<strong>on</strong> aspects of the<br />

supply chains and deals with knowledge management across distributed c<strong>on</strong>texts. This proposal<br />

c<strong>on</strong>tributes to achievement of supply chain c<strong>on</strong>tinuous improvement objectives and promotes the<br />

emergence of Learning Supply Chains.<br />

This paper has shown that the quality-related informati<strong>on</strong> capitalized through this methodology is very<br />

useful not <strong>on</strong>ly for enhancement of future quality efforts in series phase but also for enabling and<br />

improving system definiti<strong>on</strong> and suppliers selecti<strong>on</strong> during roll-out of new products. The risk and<br />

collaborativity index enhance decisi<strong>on</strong> making process at these phases and allow the synchr<strong>on</strong>izati<strong>on</strong><br />

of SCQM with the new products development projects.<br />

References<br />

Beil, D. (2010) “Supplier Selecti<strong>on</strong>”, Wiley Encyclopedia of Operati<strong>on</strong>s Research and <strong>Management</strong> Science<br />

[Electr<strong>on</strong>ic], Available: Wiley Online Library - DOI: 10.1002/9780470400531.eorms0852 [15 June 2010]<br />

Buz<strong>on</strong>, L., Ouzrout, Y. and Bouras, A. (2007) “Structurati<strong>on</strong> of the Knowledge Exchange in a Supply Chain<br />

C<strong>on</strong>text”, Paper read at the 4 th IFAC <str<strong>on</strong>g>C<strong>on</strong>ference</str<strong>on</strong>g> <strong>on</strong> <strong>Management</strong> and C<strong>on</strong>trol of producti<strong>on</strong> and Logistics,<br />

Sibiu, Romania, September.<br />

Cao, M. and Qingyu, Z. (2011) “Supply Chain Collaborati<strong>on</strong>: Impact <strong>on</strong> Collaborative advantage and firm<br />

performance”, Operati<strong>on</strong>s Managament, Vol 29, pp 163-180.<br />

Cantor D. and MacD<strong>on</strong>ald J. (2009) “Decisi<strong>on</strong>-making in the supply chain: examining problem solving<br />

approaches and informati<strong>on</strong> availability”, Operati<strong>on</strong>s Managament, Vol 27, pp 220-232.<br />

Derrouiche, R., Neubert, G. and Bouras, A. (2008) “Supply chain management: a framework to characterize the<br />

collaborative strategies”, Computer Integrated Manufacturing, Vol 21, No. 4, pp 426-439.<br />

Foster, ST. (2007) “Towards an understanding of supply chain quality management”, Operati<strong>on</strong>s <strong>Management</strong>,<br />

No. 26-2008, pp 461-467.<br />

Handfield, R., Ragatz, G., Petersen, K. and M<strong>on</strong>czka, R. (1999) “Involving Suppliers in New Product<br />

Development”, California <strong>Management</strong> Review, Vol 42, No. 1, pp 59-82.<br />

Harland, C., Zheng, J., Johnsen, T. and Lamming, R. (2004) “A C<strong>on</strong>ceptual Model for Researching the Creati<strong>on</strong><br />

and Operati<strong>on</strong> of Supply Networks”, British Journal of <strong>Management</strong>, Vol 15, pp 1-21.<br />

426


Juan Camilo Romero et al.<br />

Jabrouni, H., Kamsu-Foguem, B., Geneste, L. and Vaysse, C. (2011) “C<strong>on</strong>tinuous Improvement Through<br />

Knowledge Guided Analysis in Experience Feedback”, Engineering Applicati<strong>on</strong>s of Artificial<br />

Intelligence, Vol 24 (8), pp 1419-1431.<br />

Kamsu-Foguem, B., Coudert, T., Beler, C. and Geneste, L. (2008) “Knowledge formalizati<strong>on</strong> in experience<br />

feedback processes: An <strong>on</strong>tology-based approach”, Computers in Industry, Vol 59, pp 694-710.<br />

Knowles, G., Whicker, L., Heraldez, J. and Del Campo, F. (2005) “A c<strong>on</strong>ceptual model for the applicati<strong>on</strong> of Six<br />

Sigma methodologies to supply chain improvement”, Logistics: Research & Applicati<strong>on</strong>s, Vol 8, pp 51-<br />

65.<br />

Mavris, D. and Pin<strong>on</strong>, O. (2011) “An Overview of Design Challenges and Methods in Aerospace Engineering”,<br />

Paper read at the 2 nd Internati<strong>on</strong>al <str<strong>on</strong>g>C<strong>on</strong>ference</str<strong>on</strong>g> <strong>on</strong> Complex Systems Design and <strong>Management</strong>, Paris,<br />

France, December.<br />

Vareilles, E., Aldan<strong>on</strong>do, M., Codet de Boisse, A., Coudert, T. and Geneste, L. (2012) “How to take into account<br />

general and c<strong>on</strong>textual knowledge for interactive aiding design: Towards the coupling of CSP and CBR<br />

approaches”, Engineering Applicati<strong>on</strong>s of Artificial Intelligence, Vol 25, Issue 1, pp 31-47.<br />

Wils<strong>on</strong>, P., Dell, L. and Gaylord, F. (1993) Root Cause Analysis: A tool for Total Quality <strong>Management</strong>, ASQ,<br />

United States of America.<br />

Zhou H. and Bent<strong>on</strong> W. (2007) “Supply Chain Practice and informati<strong>on</strong> sharing”, Operati<strong>on</strong>s Managament, Vol<br />

25, pp 1348-1365.<br />

427


428


Work<br />

in<br />

Progress<br />

Papers<br />

429


430


Recording our Professi<strong>on</strong>al Development Just Became<br />

Easier: Using a Learning <strong>Management</strong> System<br />

Mercy Kesiena Clement-Okooboh<br />

University of Bolt<strong>on</strong>, Institute for Educati<strong>on</strong>al Cybernetics, UK<br />

Mkc1iec@bolt<strong>on</strong>.uk.ac<br />

Abstract: This study presents an evoluti<strong>on</strong>ary learning management system (LMS) that is used as a tool to<br />

record c<strong>on</strong>tinuing professi<strong>on</strong>al development (CPD) data. It is a critical comp<strong>on</strong>ent of Company X initiative to<br />

promote the c<strong>on</strong>tinuous improvement of its services to its internal and external clients through the enhanced<br />

skills, knowledge and competencies of its employees. The changes in the workplace learning challenged the<br />

learning and development practiti<strong>on</strong>ers to rethink new ways to improve the work based practices within the<br />

organizati<strong>on</strong> through the implementati<strong>on</strong> of processes and procedures in place to capture and analyse CPD data<br />

across different sites of the multinati<strong>on</strong>al organizati<strong>on</strong>. The mode of recording this large amount of data is as<br />

important as the learning interventi<strong>on</strong> itself and understanding the benefits of capturing large data sources in a<br />

central system is critical. The purpose of the LMS is toward c<strong>on</strong>solidati<strong>on</strong> and providing a single, comm<strong>on</strong><br />

infrastructure to manage and track learning and development initiatives across the multiple organizati<strong>on</strong> sites.<br />

Keywords: learning management system (LMS), c<strong>on</strong>tinuing professi<strong>on</strong>al development (CPD), summative<br />

evaluati<strong>on</strong>, formative evaluati<strong>on</strong>, emerging technologies, learning technologies<br />

1. Introducti<strong>on</strong><br />

This study explores how employees in a service industry resp<strong>on</strong>d to a Learning <strong>Management</strong> System<br />

tool developed to record learning hours as part of their c<strong>on</strong>tinuing professi<strong>on</strong>al development (CPD).<br />

The use of a Learning management system plays a key role in helping organisati<strong>on</strong>s cope with<br />

informati<strong>on</strong>al deluge (Kriegel, 2011), as LMS plays a critical part in helping organizati<strong>on</strong>s store large<br />

amount of persistent data safely and efficiently across different parts of the organizati<strong>on</strong>. Pina et al,<br />

2008 determine that learners place high value <strong>on</strong> usability experience. The use of a learning<br />

management system in organizati<strong>on</strong>s collates large data into a single platform that is accessible to<br />

any<strong>on</strong>e (Dabbagh and Bannan-Ritland, 2005; Ullman and Rabinowitz, 2004). As with most new and<br />

evolving technologies, the LMS is not without its limitati<strong>on</strong>s and disadvantages Ioannu and Hannafin,<br />

2008; Pina, 2007) reported that many users of LMS found it slow, c<strong>on</strong>fusing, and focused more <strong>on</strong><br />

administrative needs that learners needs.<br />

2. Literature review<br />

Learning <strong>Management</strong> System is defined as “an informati<strong>on</strong> system that administers instructor-led<br />

and e-learning courses and keeps track of learners’ progress (Brown and Johns<strong>on</strong>, 2007). Used<br />

internally by large enterprises for their employees, an LMS can be used to m<strong>on</strong>itor the effectiveness<br />

of the organizati<strong>on</strong>s’ educati<strong>on</strong> and training”. Learning technology is defined as any applicati<strong>on</strong> of<br />

technology for the enhancement of teaching, learning and assessment, an essential comp<strong>on</strong>ent in a<br />

learning technology is the ease with which the learner can interact with the system and this is termed<br />

as human computer interacti<strong>on</strong> Pierre, (2007) describes LMS as a web server based software<br />

applicati<strong>on</strong> that provides the administrative and data tracking functi<strong>on</strong>s.<br />

Szabo and Flesher (2002) advocate from a business perspective that the LMS are essentially,<br />

infrastructures that support the delivery and management of instructi<strong>on</strong>al c<strong>on</strong>tent, the identificati<strong>on</strong><br />

and assessment of individual and organizati<strong>on</strong>al learning goals, and the management of the<br />

progressi<strong>on</strong> towards meeting those goals. Gilhooly (2001) reaffirms that the functi<strong>on</strong> of LMS goes<br />

bey<strong>on</strong>d c<strong>on</strong>tent delivery to course administrati<strong>on</strong>; registrati<strong>on</strong>, tracking, reporting and skills gap<br />

analysis. Neto and Brasileiro (2007) further describe LMS as a technology necessary for supporting<br />

the educati<strong>on</strong>al needs of the informati<strong>on</strong> age.<br />

2.1 The impact of learning management system (LMS)<br />

Sibler and Foshay (2010) argue that scalable enterprise c<strong>on</strong>sumpti<strong>on</strong> of e-learning became possible<br />

through the use of LMS and its ability to plug c<strong>on</strong>tent within them. Currently, there is a rapid<br />

development in the use of learning management systems to record data in organizati<strong>on</strong>s. The use of<br />

a learning management system helps to bridged this gap by leveraging strategic learning<br />

management soluti<strong>on</strong>s to link strategic functi<strong>on</strong>s to better capture, manage, and improve the<br />

knowledge, skills and attitudes of every employee across the organisati<strong>on</strong>. LMS ensures useful and<br />

431


Mercy Kesiena Clement-Okooboh<br />

important data are captured as it takes place; and the data is available to all authorized users at any<br />

time making informati<strong>on</strong> easily accessible.<br />

N<strong>on</strong>aka (2008) advocates that an organizati<strong>on</strong>s’ ability to create, store and disseminate knowledge is<br />

crucial for staying ahead of competiti<strong>on</strong> in areas of quality, speed, innovati<strong>on</strong> and price. The use of<br />

technology to manage knowledge within an organizati<strong>on</strong> can lead to a l<strong>on</strong>g term productivity gain and<br />

to stay in business. The use of an LMS as a knowledge management tool in organizati<strong>on</strong> is very<br />

popular. Marquardt (2011) asserts that knowledge enables key stakeholders to assign meaning to<br />

large data and thereby generate informati<strong>on</strong>; having a comprehensive system in place to manage<br />

organizati<strong>on</strong> knowledge involves six sub systems namely – acquisiti<strong>on</strong>, creati<strong>on</strong>, storage, analysis<br />

and data mining, transfer and disseminati<strong>on</strong>, applicati<strong>on</strong> and validati<strong>on</strong>. Therefore, technology has<br />

become a large part of the learning functi<strong>on</strong> in organizati<strong>on</strong>s (Brand<strong>on</strong> Hall Research, 2008)<br />

Bersin and Associates (2006) asserts that organizati<strong>on</strong>s that invests in LMS have much higher<br />

efficacy and efficiency levels.This entails that LMS are the sole learning repository in organizati<strong>on</strong>s for<br />

storing data and informati<strong>on</strong>. According to Brand<strong>on</strong> Hall Research (2008) over 65% of organizati<strong>on</strong>s<br />

have an LMS, this highlights the significance of LMS as a learning tool in organizati<strong>on</strong>s. Scmidt (2003)<br />

outlines reas<strong>on</strong>s why organizati<strong>on</strong>s utilise LMS; namely reduce the cost of training, pers<strong>on</strong>alise<br />

learning c<strong>on</strong>tent and make delivery accessible to learners. LMS can faciltate the tracking of employee<br />

training records, send training notificati<strong>on</strong>s to learners, administrators and line management when<br />

required.<br />

3. Research methodology<br />

This is a case study research and it is a useful research strategy used in the dynamics of technology<br />

implementati<strong>on</strong>. McCuche<strong>on</strong> and Meredith, (1993) and Benbasat et al, (1987) advocates that case<br />

study is a valid research strategy in management informati<strong>on</strong> systems. Wils<strong>on</strong> and Woodside (1999),<br />

and Hillebrand et al. (2001) also assert case study research as a strategy that is useful for theory<br />

testing. Hence, this study employs a case study research method for explorati<strong>on</strong> and explanati<strong>on</strong> of<br />

the implementati<strong>on</strong> process of the new Learning <strong>Management</strong> System (LMS), to record data of<br />

c<strong>on</strong>tinuing professi<strong>on</strong>al development (CPD) of the employees in the organizati<strong>on</strong>.<br />

In this study, data was gathered using the formative and summative evaluati<strong>on</strong> approaches; these<br />

approaches were carried out through survey questi<strong>on</strong>naires and semi-structured interviews <strong>on</strong> a<br />

selected sample of employees and line managers from the different locati<strong>on</strong> sites<br />

4. Results and discussi<strong>on</strong><br />

4.1 Company X c<strong>on</strong>text<br />

Company X is a Energy and Utilities Service Company based in Dublin Ireland. Its parent company is<br />

a large French multinati<strong>on</strong>al with operati<strong>on</strong>s in 42 countries. The company has 411 employees in<br />

Dublin and Belfast and has been in operati<strong>on</strong>s in Ireland since 1990. Company X has a range of<br />

different portfolio’s that include energy, business development, pharmaceutical, technical, central<br />

services, human resources, alternative energy, industrial and informati<strong>on</strong> technology. Each portfolio<br />

has a director and a client operati<strong>on</strong>s manager that oversees activities in the portfolio.<br />

The company’s purpose is to “Improve the operati<strong>on</strong>al and envir<strong>on</strong>mental efficiency of utilities and<br />

energy by co-designing and delivering enhanced soluti<strong>on</strong>s in a safe, compliant and sustainable way in<br />

the Pharmaceutical, Healthcare, Industrial, Food & Beverage, ICT and Public Sectors”.<br />

4.1.1 Use of learning management system (LMS) in Company X<br />

The recording of CPD is a critical comp<strong>on</strong>ent of our initiative to promote the c<strong>on</strong>tinuous improvement<br />

of our services to our customer through the enhanced skills, knowledge and competencies of our<br />

employees. For this reas<strong>on</strong>, the organisati<strong>on</strong> decided to use our new LMS <strong>on</strong>line tool to ask<br />

employees to record their CPD data, to ensure that we are fostering and cultivating a learning<br />

envir<strong>on</strong>ment and also capturing large data in <strong>on</strong>e central system. In the past, internal classroom<br />

learning and development was m<strong>on</strong>itored and recorded through our HR database by the Learning &<br />

Development Team. However, a great deal of employee development takes place outside of the<br />

formal classroom based learning scenario and this is not always captured, recorded or evaluated.<br />

432


Mercy Kesiena Clement-Okooboh<br />

4.1.2 Benefits of using a learning management system to record CPD hours<br />

Emerging technologies, changing workplace demand are some of the factors that now challenge<br />

organisati<strong>on</strong>s to extend the scope of their practice to efficiently and effectively deliver and manage the<br />

learning experiences of their employees from a prec<strong>on</strong>ceived way of training, to <strong>on</strong>e of cultivating a<br />

learning culture and learning of individuals, teams, and organisati<strong>on</strong>s. To be able to do this,<br />

organisati<strong>on</strong>s must ensure that the learning experiences of its workforce must be accessible and<br />

easily tracked by key stakeholders. The benefits of a learning management are namely:<br />

Centralized learning envir<strong>on</strong>ment to ensure c<strong>on</strong>sistency across the organisati<strong>on</strong><br />

Tracking and reporting for enhanced performance of the learning and development team<br />

Immediate capabilities evaluati<strong>on</strong><br />

Regulatory and legal compliance<br />

4.2 What worked well?<br />

The implementati<strong>on</strong> of the LMS has created a self-directed learning am<strong>on</strong>gst employees in the<br />

organisati<strong>on</strong>. Self-directed learning can be described as self-reflective learning. Mezirow, (1985), this<br />

is similar to the process described by Argyris (1992) as double-loop learning. It is a process of<br />

recording achievement and acti<strong>on</strong> planning. According to some of the resp<strong>on</strong>ses from participants<br />

surveyed, participants were asked what worked well with the new learning management system.<br />

“The introducti<strong>on</strong> of the new LMS has enabled us to take full ownership of our learning<br />

development, we can now easily track our development regularly; the access to a<br />

pers<strong>on</strong>al CPD log will enhance our career progressi<strong>on</strong>”.<br />

The new system improved the operati<strong>on</strong> system between the learning and development team, line<br />

management and senior management teams as essential training data was captured and analyzed<br />

quickly. The additi<strong>on</strong>al add-<strong>on</strong> comp<strong>on</strong>ents of a pers<strong>on</strong>al space area allotted to individual learners in<br />

the LMS enables individuals to take c<strong>on</strong>trol of their pers<strong>on</strong>al development for recording c<strong>on</strong>tinuing<br />

professi<strong>on</strong>al development hours.<br />

4.2.1 What did not work?<br />

The number of people involved in the project slowed the implementati<strong>on</strong> process, during the<br />

implementati<strong>on</strong> stage the engineer from the system provider left the company and this meant that the<br />

new engineer had to go over the process again because there was no proper handover, same was<br />

the situati<strong>on</strong> in the organisati<strong>on</strong> as the internal project co-ordinator was out <strong>on</strong> maternity leave and the<br />

new project co-ordinator did not have a proper handover.<br />

4.2.2 What could have worked better?<br />

While some users felt the interface was dull and rigid. Siemens, (2004) noted that the LMS interface is<br />

not user friendly and should be simplified and made more intuitive.<br />

5. C<strong>on</strong>clusi<strong>on</strong><br />

This paper supports and focuses <strong>on</strong> a practiti<strong>on</strong>er perspective of a learning management system as a<br />

vehicle for recording and tracking professi<strong>on</strong>al development hours. Firstly, this study proved the<br />

importance of using the new LMS to track learning participati<strong>on</strong> and effectiveness of employees in<br />

<strong>on</strong>e system. Sec<strong>on</strong>dly, employees felt it gave them more aut<strong>on</strong>omy to take their learning development<br />

into their own hands. Thirdly, the system reduced the time spent by the learning & development team<br />

to manage, store and analyse big data from different sources across the organisati<strong>on</strong>. Lastly, the LMS<br />

steered a self-driven learning management process am<strong>on</strong>gst employees and provided data for annual<br />

statistics.<br />

The case study has highlighted the importance of a learning management system in an organisati<strong>on</strong>.<br />

The evaluati<strong>on</strong> of a learning management system requires the integrati<strong>on</strong> of different aspects such as<br />

interoperability, technical characteristics, usability, scalability, high availability, user experience, cost<br />

and effectiveness, stability and security. The findings from these approaches helped to establish their<br />

percepti<strong>on</strong>s and experiences, in relati<strong>on</strong> to the ease of use of the system to record their professi<strong>on</strong>al<br />

development data. The feedback derived from this approach was fed back to the IT department and<br />

433


Mercy Kesiena Clement-Okooboh<br />

the LMS provider to make corrective acti<strong>on</strong>s and amend some functi<strong>on</strong>alities to enable users to log<br />

their learning data easily.<br />

6. Limitati<strong>on</strong>s and further research<br />

This research study was limited to a single organisati<strong>on</strong>; further research may be c<strong>on</strong>ducted <strong>on</strong> more<br />

case studies to evaluate the impact of LMS implementati<strong>on</strong> in organizati<strong>on</strong>s. It is expected that<br />

supplementary research studies could focus <strong>on</strong> collecting more data and incorporating empirical data<br />

from large organizati<strong>on</strong>s by comparing the results with findings of this study.<br />

References<br />

Argyris, C (1992) On Organizati<strong>on</strong>al Learning, Blackwell, Cambridge, MA<br />

Benbasat, I., Goldstein, D.K., and Mead, M. 1987, the case research strategy in studies of informati<strong>on</strong> systems.<br />

MIS Quarterly, 11(3):369-386<br />

Boud, D., & Garrick, J. (1999) Understanding Learning at Work: L<strong>on</strong>d<strong>on</strong>, Routledge.<br />

Bramley, P. (2003) Evaluating Training. (2 nd ed.), Chartered Institute of Pers<strong>on</strong>nel and Development. www.cipd.ie<br />

Brown, M.I., Doughty, G.F., Draper, S.W., Henders<strong>on</strong>, F.P., McAteer, E. (1996) "Measuring learning resource<br />

use" submitted to this issue of Computers & Educati<strong>on</strong><br />

Gilhooly, M., (2001) Quality of life and real life cognitive functi<strong>on</strong>ing. Growing older programme Newsletter, 2;6<br />

Hillebrand, B., Kok, R.A.W., and Biemans, W.G. 2001, Theory-testing using case studies: a comment <strong>on</strong><br />

Johnst<strong>on</strong>, Leach, and Liu. Industrial Marketing <strong>Management</strong>, 30: 651-657<br />

Ioannou, A., and Hannafin, R., (2008). Deficiencies of Course <strong>Management</strong> Systems: Do students care?<br />

Quarterly Review of Distance Educati<strong>on</strong>, 9 (4).<br />

Johns<strong>on</strong>, A. and Ruppert, S. (2002). “An evaluati<strong>on</strong> of accessibility in <strong>on</strong>line learning management systems”, in<br />

Library Hi Tech, vol. 20 (4), pp. 441-451<br />

Kiergel, A. (2011) Discovering SQL: a hands-<strong>on</strong> guide for beginners, Wiley, 2011.<br />

Klinger et al (2010) Learning <strong>Management</strong> System Technologies and Software Soluti<strong>on</strong>s for <strong>on</strong>line Teaching:<br />

tools and applicati<strong>on</strong>s. Yurchak printing Inc., U.S.A<br />

Marquartd, M.J (2011) Building the learning organizati<strong>on</strong>: achieving strategic advantage through a commitment to<br />

learning. 3 rd Editi<strong>on</strong>, Nicholas Brealey Publishing Inc.<br />

McCutche<strong>on</strong>, D.M. and Meredith, J.R. 1993, C<strong>on</strong>ducting case study research in operati<strong>on</strong>s management. Journal<br />

of Operati<strong>on</strong>s <strong>Management</strong>, 11: 239-256<br />

Mezirow, JA (1985) A critical theory of self-directed learning, in Self-directed Learning: From theory to practice,<br />

ed S Brookfield, Jossey-Bass, San Fransisco, CA<br />

Neto Mendes Milt<strong>on</strong>, Francisco and Brasileiro, Francisco (2007) Advances in Computer-Supported Learning.<br />

Informati<strong>on</strong> Science Publishing, U.S.A.<br />

N<strong>on</strong>aka, I., and Takeuchi, H., (2008) The Knowledge creating company: How Japanese companies create the<br />

dynamics of innovati<strong>on</strong>. Oxford University Press, New York<br />

Pettigrew, A and Whipp, R (1991) Managing Change for Competitive Success, Blackwell, Oxford<br />

Pierre, S (2007) E-Learning Networked Envir<strong>on</strong>ments and Architectures. A knowledge processing perspective,<br />

Springer Publicati<strong>on</strong>s, L<strong>on</strong>d<strong>on</strong><br />

Pina, A.A., Green, S., and Eggers, M.R., (2008) Learning <strong>Management</strong> Systems: Less<strong>on</strong>s from the fr<strong>on</strong>t lines.<br />

Paper presented at the annual technology in educati<strong>on</strong> (TechEd) c<strong>on</strong>ference, Ontario, CA.<br />

Sibler, K.H, & Foshay, W.R., (2010) Improving Performance in the workplace. Volume 1: Instructi<strong>on</strong>al Design<br />

and Training Delivery. Pfeiffer Publicati<strong>on</strong>s<br />

Siemens, G (2004) Learning <strong>Management</strong> Systems: The wr<strong>on</strong>g place to start learning Retrieved 28 July, 2012<br />

from http://www.elearnspace.org/Articles/lms/htm<br />

St<strong>on</strong>er, G. (1996) “LTDI Implementing Learning Technologies”, [<strong>on</strong>line], Scottish Higher Educati<strong>on</strong> Funding<br />

Council, http://www.icbl.hw.ac.uk/ltdi/implementing-it/implt.pdf (accessed <strong>on</strong> 18th April 2012)<br />

Ullman, C., & Rabinowitz, M. (2004) Course management systems and the reinventi<strong>on</strong> of instructi<strong>on</strong>.<br />

Technological Horiz<strong>on</strong>s in Educati<strong>on</strong> Journal Retrieved 28 July, 2012 from<br />

http://www.thejournal.com/articles/17014<br />

Wils<strong>on</strong>, E.J. and Woodside, A.G. 1999, Degrees-of-freedom analysis of case data in business marketing<br />

research. Industrial Marketing <strong>Management</strong>, 28: 215-229<br />

434


Is More Data Better? Experiences From Measuring<br />

<strong>Academic</strong> Performance<br />

Harald Lothaller<br />

University of Music and Performing Arts Graz, Graz, Austria<br />

harald.lothaller@kug.ac.at<br />

Abstract: At University of Music and Performing Arts Graz (Austria), we have developed and implemented a<br />

comprehensive <strong>on</strong>line system to measure artistic and scientific performance in 2007. Since the roll-out in 2008,<br />

more than 20.000 entries are made by the staff members. The main topic in the beginning was to make the staff<br />

members use the new tool. Now, we have deal with different topics: We have to keep them motivated to enter<br />

further data. We have to handle the big amount of data. And we have to increase benefits from collecting data for<br />

staff members, departments, and the university. The paper presents some of our approaches to generate these<br />

benefits and thereby also come up to the other requirements. For instance, an easy-to-handle reporting and<br />

exporting soluti<strong>on</strong> is now available for pers<strong>on</strong>s and departments. Indicators for quality management and<br />

evaluati<strong>on</strong> purposes shall be derived from the tool in the upcoming future. Administrative work load analyses<br />

based <strong>on</strong> out tool influence the human resources’ allocati<strong>on</strong> to departments. In any case, there is a str<strong>on</strong>g need<br />

to reduce complexity and we must not spread the whole data set over different purposes. We c<strong>on</strong>clude that more<br />

data might be better with regard to reporting, rankings, or funding, but also raise problems that have to be solved.<br />

Keywords: <strong>on</strong>line tool, evaluati<strong>on</strong>, quality management system, artistic performance, research performance,<br />

practical experiences<br />

1. Introducti<strong>on</strong><br />

At University of Music and Performing Arts Graz (Austria), we have developed and implemented a<br />

comprehensive <strong>on</strong>line system to measure performance that is representing the wide-spread range of<br />

activities of our artists and researchers in particular and other staff members too (presented at ECIME<br />

2009). Since the roll-out in 2008, more than 20.000 entries are made and data are mainly used for<br />

statistics and reporting. Now we have reached the point to raise the questi<strong>on</strong>s of (1) how to keep the<br />

staff members motivated to enter data, (2) how to handle the big amount of data as well as (3) how to<br />

increase benefits from collecting t<strong>on</strong>s of data aside of fulfilling the reporting that is legally required for<br />

Austrian universities.<br />

2. Work in progress<br />

The background of questi<strong>on</strong> 1 was completely different at the time of roll-out when we had to<br />

c<strong>on</strong>vince the staff members of using the new system and entering data. But meanwhile, most people<br />

have entered data and some seem to get tired of entering additi<strong>on</strong>al data. We have seen a decrease<br />

in the number of entries last year as compared to the years before. This might be due to the amount<br />

of data already entered and the feeling that <strong>on</strong>e entry more or less is negligible in a 20.000-entries<br />

system.<br />

Questi<strong>on</strong> 2 is associated with quality assurance of data. The more narrow the variety and the lower<br />

the number of entries, the easier it is to have an eye <strong>on</strong> the single entries and to assure the<br />

meaningfulness of data. In the beginning, we had internal “peer reviewing” within the staff when<br />

pers<strong>on</strong>s were aware that their colleagues would inspect their entries. But now, pers<strong>on</strong>s might assume<br />

that even waste entries could slip through the peers’ eyes within the whole system.<br />

The answers to questi<strong>on</strong> 3 hopefully become the soluti<strong>on</strong>s for the two other questi<strong>on</strong>s too. We have to<br />

further increase benefits from our existing system for single pers<strong>on</strong>s, departments, and the university.<br />

We present some of our respective activities below.<br />

The initiative reas<strong>on</strong> for developing our tool was the legal requirement of reporting performance and<br />

activities of the university staff members in the Intellectual Capital Report (ICR) to the ministry of<br />

science and to the public <strong>on</strong>ce a year. Thus, we developed a system to measure a widespread range<br />

of different kinds of activities and tried to make our staff members entering every achieved activity.<br />

We count all of them and the activities are quantitatively presented <strong>on</strong> a highly aggregated level in<br />

several tables in ICR. Qualitative informati<strong>on</strong> can <strong>on</strong>ly be derived from the heading of the tables or<br />

some subcategories within the tables, but this is usually <strong>on</strong>ly informati<strong>on</strong> about the kinds of activities,<br />

but not their quality itself. Additi<strong>on</strong>al internal reports corresp<strong>on</strong>d with the ICR approach and present<br />

highly aggregated data differentiated by our university’s departments every six m<strong>on</strong>th. This internal<br />

435


Harald Lothaller<br />

informati<strong>on</strong> is somewhat closer to the staff, but still suffering from a lack of qualitative informati<strong>on</strong>.<br />

Both ways of reporting fulfill their purpose of providing updated amounts of activities, but cannot go<br />

behind that.<br />

Providing qualitative informati<strong>on</strong> goes al<strong>on</strong>g with splitting the whole data into manageable parts. This<br />

splitting is d<strong>on</strong>e in reference to pers<strong>on</strong>s, to departments, or to the kinds of activities. Splitting in<br />

reference to pers<strong>on</strong>s leads to an individual list of the data entries. In the beginning of 2012, we have<br />

developed a query tool to enable each pers<strong>on</strong> itself to generate this pre-designed list <strong>on</strong>line and to<br />

both view <strong>on</strong> screen and save as editable file. The query tool allows selecting the time period and the<br />

kinds of activities to be shown. Thus, every pers<strong>on</strong> can now extract her own list of activities and<br />

further use for purposes like self-report preparati<strong>on</strong>, a list of publicati<strong>on</strong>s, or others <strong>on</strong> the individual<br />

level. Splitting in reference to departments leads to a list of data entries of an organizati<strong>on</strong>al unit. One<br />

the <strong>on</strong>e hand, we offer the query tool to the heads of departments also for generating list <strong>on</strong> this level.<br />

This might be useful within evaluati<strong>on</strong> processes c<strong>on</strong>cerning the unit, parts of it, or pers<strong>on</strong>s, for <strong>on</strong>eto-<strong>on</strong>e<br />

performance reviews with staff members of the department, or others <strong>on</strong> the organizati<strong>on</strong>al<br />

level. On the other hand, we are now developing a possibility for transferring data from our Oraclebased<br />

tool into typo3-based websites of departments. The departments have different requirements<br />

as well as artistic and scientific emphases. One department with a str<strong>on</strong>g scientific emphasis <strong>on</strong>ly<br />

wants to provide the publicati<strong>on</strong>s <strong>on</strong> the websites, for instance. But this would not fit other<br />

departments with artistic emphases. Therefore, it is necessary to offer different intersecti<strong>on</strong>s<br />

corresp<strong>on</strong>ding to their needs. Anyway, the data must be updated c<strong>on</strong>tinuously to keep the websites<br />

up-to-date and the resp<strong>on</strong>se times of the websites must be very short, so the amount and format of<br />

data that has to be suchlike that it can be transferred to and processes within the websites efficiently.<br />

Splitting in reference to the kinds of activities is c<strong>on</strong>nected to the differences between the<br />

departments and d<strong>on</strong>e within this project. Furthermore, it leads us to quality management purposes<br />

that shall be supported by means of the data from our tool in the future.<br />

In anticipati<strong>on</strong> of an instituti<strong>on</strong>al quality audit of our university by an external agency, we are<br />

developing a comprehensive quality management system. Therein, we are including our previous<br />

activities in the field of quality management. But we also have to bring together different activities that<br />

have been somewhat isolated of each other before as well as fill some gaps in our system. One<br />

current gap is the systematic evaluati<strong>on</strong> of the artistic and scientific work based <strong>on</strong> our university’s<br />

quality aims. The university management and deputies of the faculty staff together are breaking down<br />

each single aim to both qualitative and quantitative indicators to be observed systematically and<br />

periodically. And several of these indicators should be derived from our tool as we do not <strong>on</strong>ly<br />

measure the number of activities, but each entry is accompanied by useful informati<strong>on</strong> c<strong>on</strong>cerning the<br />

respective activity. The tricky thing is that there are no given standards in evaluati<strong>on</strong> of artistic<br />

activities in particular but some scientific disciplines too. What would be an indicator for our aim to<br />

preparing young artists to be competitive in their community based <strong>on</strong> an aut<strong>on</strong>omous artistic<br />

identity? Or what would be an indicator for our aims to have excellent artistic and scientific work d<strong>on</strong>e<br />

by our staff members that is internati<strong>on</strong>ally visible and remarkable? Each pers<strong>on</strong> or each artistic and<br />

scientific field might have a different definiti<strong>on</strong> of what is excellent, of what is visible and remarkable,<br />

or of what is an aut<strong>on</strong>omous artistic identity. But we need objectivity, comparability, and acceptance<br />

of our indicators across the whole university and the possibility of internal benchmarkings over time<br />

periods. This will be a project for the end of this year and bey<strong>on</strong>d. There are a lot of ideas about what<br />

might be possible additi<strong>on</strong>ally afterwards like automatically matching pers<strong>on</strong>s for evaluati<strong>on</strong> and<br />

reviewing processes or bringing together staff members from different departments for completely<br />

new interdisciplinary work based <strong>on</strong> analyses of the indicators. But it will be a l<strong>on</strong>g way to go before<br />

this will come true.<br />

In some projects, our tool already proved to be helpful although their aims were not in its primary<br />

focus. In the area of human resources, a project tried to find out about the work load of the<br />

administrative staff members in the academic departments. These pers<strong>on</strong>s support the artistic and<br />

scientific as well as teaching staff in fulfilling their duties. Am<strong>on</strong>g others, several work load indicators<br />

are periodically derived from our tool to evaluate the administrati<strong>on</strong>’s work load and compare between<br />

the departments as well as over time periods. Resources are allocated based <strong>on</strong> these facts. As<br />

departments that are proven under-staffed can get additi<strong>on</strong>al resources and better administrative<br />

support, all staff members of the department can see the benefits from the project and our tool.<br />

Another project does not use our data, but the technical basis of our tool for its purposes. With few<br />

adapti<strong>on</strong>s, the event unit of our university is now able to collect all necessary informati<strong>on</strong> of nearly<br />

436


Harald Lothaller<br />

1.000 different events per year. All entries are inserted automatically into their <strong>on</strong>line calendar of<br />

events and announcement <strong>on</strong> websites. In additi<strong>on</strong>, event data can be easily analyzed for time peeks<br />

or room use for instance. We assume that similar projects in other areas might follow.<br />

3. Less<strong>on</strong>s learnt<br />

We have learned that we are not able to increase benefits by just spreading the whole data set over<br />

various purposes. There is a str<strong>on</strong>g need to reduce complexity when generating benefits because<br />

neither individuals nor units are able to deal with the amount of entries and the variety of activities of<br />

our measurement tool for their practical needs. So we often have to select rather few aspects from the<br />

comprehensive approach of measuring performance of university staff members. Therefore, we have<br />

seen again a need for a bottom-up approach which means to go back to the individuals and units that<br />

had been involved in the development of our tool some years ago. Back then, they had been asked to<br />

name various kinds of activities to be measured as our aim was to represent all our staff members’<br />

work in our tool. C<strong>on</strong>sequently, we added more and more aspects to our tool. This time, we asked<br />

them to select parts from the tool that are relevant for their specific purposes to be fulfilled.<br />

C<strong>on</strong>sequently, we pick out some aspects of our tool now. Some aspects are used several times and<br />

for different purposes. But we see now that some aspects are not used at all, not even by those who<br />

had asked for them some years ago. Aside, some discussi<strong>on</strong> about academic quality, quality<br />

management, and evaluati<strong>on</strong> have resulted <strong>on</strong> a more general level and distributed these thoughts<br />

within the university.<br />

4. C<strong>on</strong>clusi<strong>on</strong><br />

Of course, in reporting to the public, counting more and more activities is alluring. University rankings,<br />

benchmarking approaches, or funding discussi<strong>on</strong>s often refer exclusively to the counted performance<br />

but not its quality. But aside, the practical experiences have made us rethink in some ways. Having a<br />

clear focus instead of a broad approach as well as highlighting certain aspects instead of looking for<br />

increasing counts is needed in additi<strong>on</strong>. We will not rebuild and shrink our comprehensive system for<br />

sure. Anyway, we maybe would take a slightly different approach now as we have seen that more<br />

data can be nice and give opportunities, but can also raise problems of manageability and are not<br />

necessarily better.<br />

437

Hooray! Your file is uploaded and ready to be published.

Saved successfully!

Ooh no, something went wrong!