TU/e notebook program - Technische Universiteit Eindhoven

w3.tue.nl

TU/e notebook program - Technische Universiteit Eindhoven

TU/e notebook program

Evaluation

University of Groningen

Marian de Vaan 1673645


Supervisors

Evaluation of the TU/e notebook program

Prof.dr. E.W. Berghout

Faculty of Economics & Business

University of Groningen

Dr.ir. R.P. Waterham

Director Dienst ICT

Eindhoven University of Technology

Marian de Vaan

University of Groningen

MSc BA Specialization General Management

MSc Business Administration

June, 2008

m.j.d.vaan@tue.nl


ACKNOWLEDGEMENT

This report presents the findings of a research on IT evaluation methods, which was

conducted by a case study on the notebook program of Eindhoven University of Technology.

The research was part of my study at the University of Groningen and had to lead to my

graduation. I was asked by the director of the IT Service organization of Eindhoven

University of Technology to perform an evaluation on the notebook program of the university

and performed the research between December 2007 and June 2008. Although Business &

ICT was not a part of my education program, I took the opportunity to perform the evaluation

because I, like my client, believe that it is necessary to evaluate any IT/IS system or

infrastructural facility that is of significant influence on the organization.

The assignment took place under the auspices of the director of IT Services dr. ir. R.P.

Waterham, who in spite of his busy agenda always made time free when I needed some extra

information. He also introduced me to his colleagues at other universities I researched for the

benefit of the evaluation. I would like to thank him for his cooperation. I also would like to

express my gratitude to the directors of the IT service organization of the universities I visited

for this research. All the people at the University of Eindhoven that were willing to help me

out with the questions I had regarding the effects of the notebook program deserve my

gratitude. All my colleagues IT coordinators about the effects on their jobs and activities, and

the employees of the various departments that spoke to me about the effects on the education

program. Moreover great support I received of Francis and Steven of the Centre for

Communication, Language and Technology; they made it possible to fulfill one of my wishes

in this assignment: to write it in English, I thank them for that support.

Prof.dr. Egon Berghout was my supervisor of the University of Groningen. I couldn’t have

wished for anyone better; he is one of the authorities in the field of IT evaluation. His

knowledge helped me to achieve a good result with the evaluation and supported the process.

Finally I would like to thank my colleagues, family and friends, and fellow students for their

support in various ways. My special thank goes to my husband and buddy Piet, without his

encouragement and moral support I would never have come so far in this relatively short

period of time.

Before you lay the final report of the results of my findings for this assignment, I hope you

enjoy reading it.

June 2008, Marian de Vaan

2/85


MANAGEMENT SUMMARY

This report presents the research that has been conducted to determine whether a chosen IT

evaluation method that is developed as an IT investment method is also suitable to perform an

ex post evaluation. The evaluation itself was executed on the notebook program of Eindhoven

University of Technology. The underlying research question was:

What is the result of the evaluation of the notebook program at Eindhoven University

of Technology after the selection of a suitable method?

Before a choice for an evaluation method could be made, a literature research was conducted,

in which various categories of IT evaluation methods were investigated. The result of the

literature research was, besides the acquirement of basic knowledge about these categories,

the choice of New Information Economics (NIE) as the method to be used to perform the

evaluation. After that decision and before the actual performance of the evaluation the method

was studied thoroughly, and the elements that were applicable for an ex post evaluation were

selected. As a result of that selection the Alignment Practice and the Performance

Measurement Practice were utilized to perform the evaluation.

The notebook program of Eindhoven University of Technology consists of a notebook,

however, also includes the accompanying IT infrastructure, applications like an electronic

learning system to support the education programs, supportive services like service desks, and

the provisioning of notebooks to students with a financial contribution of the university.

Although the idea existed that the notebook program is an expensive way to fulfill the

strategic intention of providing students with IT facilities any place and any time, the

evaluation indicates otherwise. The results of the financial evaluation indicate that, because of

the use of notebooks to fulfill the strategic intention, a cost reduction of 1,675,000€ occurs.

Besides this cost reduction on hardware, the notebook program attracted more students in the

first two years of its implementation. That number of first-year students did not decrease, and

increases contribution from the authorities with 7,539,075€ for the first two years after

implementation of the program..

So the internal evaluation showed that of the two alternatives, desktops and notebooks, the

solution with notebooks was the beneficial one. To compare this with other universities, a

benchmark was performed between the universities of Eindhoven, Delft, Groningen, and

Utrecht. The benchmark also showed that the use of notebooks as part of the integral IT

infrastructure was beneficial.

3/85


Due to the knowledge students acquire by managing and operation their own notebook the

number of IT support staff was lower compared with the other universities, as a result

Eindhoven supports more workplaces per fte support staff.

Also regarding the other effects the evaluation was positive. Even though communication by

electronic means is common nowadays, the evaluation indicated that the use of those means

did occur earlier in time and increased further because notebooks were part of the integral IT

facilities. To investigate this effect a small sample of TU/e students was asked to fill in a

questionnaire and teachers were asked questions regarding communication during interviews.

The third aspect measured was the competitive advantage Eindhoven University of

Technology was gained from the notebook program. The internal evaluation indicated that in

the first two years after the implementation of the notebook program a significant increase in

first-year students occurred, respectively 13% and 17%. But to determine if it was really

ascribable to the notebook program, and not a coincidence of simply more students enrolling

in a technical education program a benchmark with the other technical universities was

performed. This benchmark was restricted to the number of first-year students, and indicated

that at the same time there was no increase in first-year students neither at Delft University of

Technology nor at the Twente University. The increase of the first-year students in the first

two consequentive years is therefore associated to the notebook program. This is however not

sustainable strategic advantage according to Porter’s definition.

The New Information Economics method demonstrated to be suitable for the ex post

evaluation of the notebook program of Eindhoven University of Technology, but some

recommendations for adaptation are made.

The method is extended, which makes it applicable for various evaluations, ex ante and ex

post. Because the various Practices can be used separately, it is not necessary to apply all

practices in a given situation. This means that the extensiveness of NIE is not a disadvantage

for the evaluation of a less complex situation. Within the Practices itself there are enough

possibilities to adapt the elements on which the evaluation is to be performed.

The only recommendation is to remove the unnecessary elements and practices, and compose

a method especially for ex post evaluation appliances. It would make the method less

complex in case of a straightforward ex post evaluation.

4/85


TABLE OF CONTENTS

Acknowledgements ....................................................................................................2

Summary .............................................................................................................3

1. Introduction

1.1. Context .........................................................................................................8

1.2. Objective of the research.................................................................................8

1.3. Description TU/e

1.3.1. General information .........................................................................9

1.3.2. Description of the supportive bureaus involved ................................10

1.4. The Notebook Program

1.4.1. Introduction......................................................................................11

1.4.2. Purposes and goals...........................................................................12

1.4.3. The annual process...........................................................................13

1.5. Research design

1.5.1. Introduction......................................................................................15

1.5.2. Research questions...........................................................................15

1.5.3. Research methodology .....................................................................16

1.6. Outline of the thesis........................................................................................17

2. Literature research on IT evaluation

2.1. Introduction ....................................................................................................18

2.2. Objective ........................................................................................................18

2.3. Definitions......................................................................................................20

2.4. Evaluation methods

2.4.1. Introduction......................................................................................21

2.4.2. Evolution of IT evaluation methods..................................................21

2.4.3. Categories of IT evaluation methods ................................................23

2.5. Problems and deficiencies in IT evaluation .....................................................28

2.6. Conclusions ....................................................................................................29

2.7. Selection of an evaluation method for the notebook program .........................29

5/85


3. Evaluating the notebook program TU/e

3.1. Introduction ....................................................................................................32

3.2. Strategic Intentions .........................................................................................32

3.3. The Alignment Practice ..................................................................................33

3.3.1. Part I Strategic Alignment ................................................................34

3.3.2. Part II Internal IT Alignment............................................................37

3.4. Performance Measurement Practice ................................................................38

3.4.1. Part I Cost Performance ...................................................................38

3.4.2. Part II Quality and Service Levels ....................................................42

3.4.3. Part III Process Measurement...........................................................44

3.5. Effects on communication...............................................................................44

3.6. Competitive advantage ...................................................................................46

3.7. Analysis .........................................................................................................49

3.7.1. Strategic alignment...........................................................................49

3.7.2. Performance measurement ...............................................................50

3.7.3. Communication................................................................................51

3.7.4. Competitive advantage.....................................................................53

4. Benchmark between four universities

4.1. Introduction ....................................................................................................54

4.2. Objective of the benchmark ............................................................................54

4.3. Universities included in the benchmark...........................................................55

4.4. Personnel benchmark .....................................................................................59

4.5. Financial benchmark.......................................................................................61

4.6. Analysis .........................................................................................................62

5. Conclusions and recommendations

5.1. Introduction ....................................................................................................64

5.2. Results of the evaluation.................................................................................64

5.3. Results of the benchmark................................................................................66

5.4. Answering the research questions

5.4.1. Introduction......................................................................................67

5.4.2. Answering the sub research questions ..............................................67

5.4.3. Answering the main research question..............................................70

5.5. Recommendations regarding the used method.................................................70

6. Discussion and reflection.......................................................................................72

6/85


Appendix A References............................................................................................74

Appendix B List of figures and tables.......................................................................80

Appendix C Questions and results survey communication ........................................81

Appendix D List of interviewees...............................................................................85

7/85


1 INTRODUCTION

1.1 Context

Eindhoven University of Technology is in pursuit of a more intensive use of modern

information and communication technology in its education, research, and activities in the

domain of social services. In the plans for the coming years explicit room will be created for

the development of matters like computer-supported education, e-learning and the use of new

media. Especially because TU/e students are known for their frequent use of the internet, the

TU/e will stimulate more applications of the internet in student facilities.”

Source: Compete and Collaborate, Strategic Blueprint of Eindhoven University of Technology, March 1997

This strategic statement was the beginning of what once was called the notebook project, and

what has become the current notebook program at the TU/e.

Although the notebook program now has reached its tenth anniversary, there has never been a

serious attempt taken to evaluate its results. Did the notebook program really bring what was

expected at first? Did any other benefits occur that were not thought of at that time?

Because there was no extended evaluation, it is necessary to quantify the results of the

notebook program which gives the opportunity to provide a well-founded answer to that

question. Within the scope of this research the focus will be on the financial aspects of that

question. Furthermore, research is conducted on the effects of the use of notebooks in the

education programs, and on the possible advantages for Eindhoven University of Technology

regarding competitive issues. Especially to have an indication regarding the latter aspect it is

necessary to perform a benchmark with other, comparable, organizations.

1.2 Objective of the research

The objective of this research is to come to a well-founded statement about the

appropriateness of a chosen method to evaluate an IT concept that is comparable to the

notebook program at Eindhoven University of Technology. The chosen evaluation method

will therefore be conducted on the notebook program.

8/85


1.3 Description TU/e

1.3.1 General information

The mission of the Eindhoven University of Technology:

“The TU/e strives to be a research driven and design directed Technology University

of an international level, which aims to provide young people with an academic

education within the domain of engineering science and technology”.

The TU/e consists of 9 departments; together they provide 12 Bachelor and 26 Master’s

education programs. Furthermore, there are 4 Master’s programs to obtain a first-degree

teacher qualification, and 10 postgraduate designer programs, which are being provided in

cooperation with the technical universities of Delft and Twente.

In total there are at the moment 7043 students, 160 fulltime professors, and several guest and

part-time teachers. In addition the TU/e employs 639 PhD students. Next to the scientific

personnel there are 1169 support staff members.

fig.1.1 Organization Chart of Eindhoven University of Technology, source: website TU/e

9/85


1.3.2 Description of the support organizations involved

Although the name of the project and the program that provides for the use of Information

Technology in the education programs is “the notebook program”, it is much more than what

the name implies. It suggests that the main purpose of the notebook program is to equip the

students with a mere notebook. Because that would not lead to the goals of the notebook

program, from the beginning, it was clear that it was not a project or program of the IT service

organization of the university. It was not something that the bureau student administration

was going to introduce either. Both bureaus are facilitating organizations, both very needed to

implement the notebook program. But the main effort was expected from the departments.

They had to make sure that the notebook was integrated in the education programs, so that the

main objectives could be reached.

The student administration (STU) and the IT services organization (ICT Services) are

responsible for the implementation on the administrative and technical side of the notebook

program. The student administration sees to it that every potential student receives al the

information he needs about the notebook program. Within the enrolment for an education

program at the TU/e there is a part that allows the student to participate in the notebook

program. STU also takes care of the contracts a student has with the TU/e regarding the

notebook program. Because of that important administrative role, and because STU has a

direct link to the departments for education matters (for instance as owner of the education

information system, which contains information like college schedules) the role of notebook

coordinator is positioned in that bureau. His responsibility is to lead the yearly process of

choosing, purchasing, and delivering the notebook to the new students. In that function he

also manages the budget of the notebook program, and maintains the strategic contacts with

the service provider and the manufacturers of notebooks.

The IT service organization is facilitating, and has an advisory role in the decision on the

specifications of the notebook hardware. For a further description on the process for the

decision on which hard- and software are needed for the each new generation students please

refer to the description of the notebook program further on in this chapter.

10/85


1.4 The notebook program

1.4.1 Introduction

The notebook program at the TU/e was initiated as a result of a decision in the Strategic

Blueprint that was published in 1997. One statement in that Strategic Blueprint is the

intention to make sure that the TU/e is an innovative university that educates its students in

the highest possible way. One of the goals of the notebook program therefore is to provide

students with the skills they need to perform excellent in their future positions. In the case of a

technical university that means that the students need to know how to work with computer

applications that are used in technical research, like applications that allow simulations of

dangerous chemical experiments, and tools to help them to design innovative buildings. To

obtain that goal it is necessary for students to have the opportunity to use computer systems

that are available for them whenever they need them. That means that there have to be a large

number of systems available for students. In the early talks about the notebook program, the

necessity of that large number was becoming clear for most of the departments. For some of

them, for instance Electrical Engineering, the notebook program became very important. They

had no computer rooms with public computers for students, like other departments did. In

their case they still send their students to public places, like instruction rooms of the central IT

services. But students could not work there because that was a public space that was also used

for instructions and lectures on IT applications. So especially in Electrical Engineering and

other departments without their own computer rooms the call for other solutions became

louder.

Space was another possible problem that could be solved by the introduction of the notebook

program. If the departments had to invest in computer systems, where would they place them?

When you want to obtain the goal of the university in education with IT means, you have to

be able to do so. But with all these PC’s and all the space they needed, there would be a great

loss of space in the buildings. If you fill up a room with computers, you can not, or almost can

not, use it for other purposes like conferences or lectures (those without computers).

With the mindset to all these possible problems and arguments, the Executive Board of the

university ordered the development of a solution that would be feasible regarding many

aspects.

It was clear that the decision to implement the notebook program could be a step forward in

the integration of IT to support the mission statement. The performance of the notebook

program, be it good or bad, would have a major impact on that statement, and would have

major consequences for the university.

11/85


1.4.2 Purposes and goals

When a student makes use of the notebook program he will be provided with a notebook at

reduced costs. The student does not only receive a notebook (hardware) but also the necessary

accessories, support with the use of the notebook, and a warranty and insurance for five years.

With those conveniences the student can use a properly working notebook for (the greater part

of) his education.

Although making use of the notebook program is not obligatory, it has in recent years almost

been impossible to study at the TU/e without a notebook. Many of the education programs are

supported with the use of computer programs. For instance, there is the use of AutoCAD for

making technical drawings and the use of computer programs to simulate chemical processes.

For that purpose the student notebooks are fully equipped, not only regarding the technical

specifications of the notebook itself, but also with the applications that are needed for the

education program.

The notebook is also used for communication with teachers and fellow students; every student

has a computer account and a mailbox from the moment he enrolls in an education program at

the TU/e. That computer account can also be used for the electronic learning environment in

which students for instance can enroll for an exam.

With all of the above, the notebook has become a Virtual Study Place: at any location on the

university grounds the student can use the IT facilities that are needed for his education, and

he can continue to work on the train, in his room, or anywhere else he pleases.

Through the use of notebooks by students another advantage arose: standardization. The

notebook program forced all employees, teachers and supporting staff, to work with the same

applications. Mail is a very good example; before the notebook program several mail services

were used at the university, with the problems that you could expect: problems with

compatibility, increased management costs, and the need for more support personnel. The

same is imaginable for other applications, for instance word processing applications. A choice

had to be made for one platform. It meant that, after a while in which the investments of the

past were depreciated; a cost reduction took place for the kind of IT infrastructure. Not only a

reduction in the various computer systems and applications, but also in the costs of

maintenance and management.

With all of this there had to be a connection between the mission and the use of IT. The

notebook program was to deliver a certain education level to the graduates of the university,

especially concerning the use of IT in their future positions.

12/85


Therefore all the aspects of the notebook program, including the specifications of the

notebook itself, had to be focused on that goal. That implied that the departments had to have

a greater say in the process of choosing the notebook and all the other IT facilities than the IT

service organization. The latter statement is embedded in the annual selection process that is

described in the following section.

1.4.3 The annual process

The embedding of the use of IT in the education programs is an ongoing process. The

continually growing demands of the applications that are needed, and the increasing

development of new technologies compel the TU/e to an annual choice of the required hard-

and software for every new generation of first-year students. Therefore there is a committee of

delegates from every department, supplemented with specialists from STU and ICT Services

that make a choice regarding the new notebook model. The delegates of the departments stand

for the requirements that the departments need for the education programs. That means that

they make sure that the applications that are needed in the education program are feasible on

the system configuration of the new notebook. Specialists of STU and ICT Services have an

advisory role regarding administrational, hardware, and infrastructural issues, but they do not

have a say when it comes to making a choice.

Each year, at the beginning of February, the notebook coordinator asks the delegates of the

various departments what the requirements for the coming generation of students will be.

They in turn consult the teachers and other educationalists of their department what they think

is necessary for the coming years. The delegates then, together with the specialists of ICT

Services and with support of the service provider that has greater knowledge of the market,

formulate the minimum specifications a notebook has to meet, to fulfill the needs of the

education program for the next couple of years. The next step in the process is the request to a

selected group of notebook manufacturers to make a special offer, based on the specifications.

The selection of the manufacturers is done preceding the request for an offer, by inviting the

manufacturers to inform them of the procedure and the environment in which the notebooks

have to be used. These informal meetings are with a part of the selection committee, with the

main goal of informing the manufacturers so that they can form an idea of what is needed and

produce the best possible proposal.

13/85


After the proposal is received, together with a testing machine of the offered kind, the offers

are weighed against each other and the machines are thoroughly tested on various aspects.

With the results of the test and the conclusions of the weighing of the proposals, the delegates

of the departments decide which notebook is to be delivered to the new first-year students.

Between this decision and the actual delivery to the students much has to be done. Specialists

from both the departments and ICT Services produce an installation image for each

department. The image contains all the applications a student needs in the first year of his

education program.

In the meanwhile new students enroll in an education program; most of them participate in the

notebook program, so the employees of STU make sure that a notebook is reserved for them.

Further on in time approximately 1500 notebooks arrive at the location of the service

provider. They place the right image on the notebook that is reserved for a particular student

and make sure that when he connects to the campus network everything works as it should

work. They also pack up the accessories that are being provided to the student, such as a

separate keyboard and notebook standard for RSI prevention, and store the whole package

until the day of delivery. The figure below is the timeline for the process from specification

until delivery in 2008.

Fig. 1.2 Timeline notebook program delivery 2008

14/85


1.5 Research design

1.5.1 Introduction

“The Eindhoven University of Technology is pursuing a more intensive use of modern

information and communication technology in its education, research, and activities in the

domain of services. In the plans for the coming years explicit room will be made for the

development of matters like computer-supported education, e-learning and the use of new

media. Especially because TU/e students are known for their frequent use of the internet, the

TU/e will stimulate more applications on the internet in student facilities.”

Source: Competing and collaboration, Strategic Blueprint of Eindhoven University of Technology, March 1997

This strategic statement was the beginning of what once was called the notebook project, and

has since become the current notebook program at the TU/e.

Although the notebook program now has reached its tenth anniversary, there has never been a

serious attempt taken to evaluate its results regarding the initial goals. Did the notebook

program really bring what was expected at first?

Because there was no extended evaluation, it is necessary to quantify the results of the

notebook program to give a well-founded answer to that question.

1.5.2 Research question

The objective of this research is summarized in the following research question:

What is the result of the evaluation of the notebook program at Eindhoven University

of Technology after the selection of a suitable method?

To come to a well-founded answer to the research question it is necessary to answer the

following sub-questions:

• What are the existing evaluation methods?

• What comprises the notebook program at the TU/e?

• What are the objectives that were intended with the notebook program?

• Which method is the most suitable for the evaluation of the notebook program?

• What is the result of the evaluation?

• How can the used method be evaluated?

• Are there recommendations to adapt the chosen method to make it more suitable?

15/85


1.5.3 Research methodology

Research type

The first part of the research is a literature study aimed at selecting the most suitable method

for the evaluation. IT evaluation methods can be divided into four classes (Berghout en

Renkema, 2005):

• financial methods

• multi-criteria methods

• ratio methods

• portfolio methods

The result of the literature study is a selection of one or more methods from one or more

categories, with a possible combination of parts of methods to come to the most suitable

method for the evaluation of the notebook program. To obtain a full understanding of the

feasibility of the chosen methods, not only the theory of the methods is studied, but also case

studies that have been conducted using various methods.

Research design

The major part of the research is desk research. Desk research exclusively uses documents

and is of course practiced in the literature study (de Leeuw, 2003). But also in the case study,

the evaluation of the notebook program, the greater part of the data is extracted from analysis

of policy documents, reports, and financial statements.

The objectives of the notebook program can be extracted from the policy documents. To

verify the objectives there are interviews with the policy makers and participants of the

notebook project. With this verification, the determination of the objectives in the notebook

program is obtained through triangular research (Jonker and Pennink, 2004).

Data on the effects of the notebook program are also extracted from documents, i.e. by means

of desk research. Next to that interviews with various people will be conducted to try to

retrieve the effects of the use of notebooks in the education program, especially regarding the

communication between teachers and students, and among students in the education process.

The financial effects are obtainable through the different financial reports on the TU/e. To

obtain the data for the benchmark between the selected universities interviews will be held

with the policy makers of the IT Service organizations of those universities. More general

information, like student numbers and university budgets will be extracted from the websites

of the participating universities.

16/85


The side effects, for instance the adaptation of processes and changes in the organization of

the university, are not easy to quantify. Even when objectives can be clearly defined, it is

almost impossible to define clear criteria to measure them, to arrive at a well- founded

statement about the success or failure of the notebook program. Therefore, and due to the

limited amount of time available for this research, these side effects will not be part of this

research.

1.6 Outline of the thesis

In chapter one you will find the context in which this research took place. It is necessary to

have knowledge of the organization of Eindhoven University of Technology, and the

essentials of the notebook program to have a proper idea of the context of this evaluation.

Next to that you will find the research design and method.

Chapter two presents the literature research and its findings. The arguments that have lead to

the choice of a method are also stated in this chapter.

Chapter three is dedicated to the results of the evaluation of the notebook program. In this

evaluation the financial aspects are researched, and an attempt was made to designate the

effects of the use of notebooks in the education program as well. At the end of chapter two

you will find the analysis of the results of the research.

In chapter four the findings are given of the benchmark that has performed to underpin the

results of the evaluation. It is almost impossible to perform a proper evaluation without

comparing the results with the data of other, more or less comparable, organizations. I chose

three universities for various reasons. Those reasons, as well as a short description of the

universities, and of course the findings of the benchmark, are described in this chapter.

Chapter five contains the conclusions and recommendations that resulted from the analysis of

both the evaluation and the benchmark. In addition you will find my recommendations on the

use of the chosen method for evaluating a project similar to the notebook program.

Finally chapter six contains the discussion on the evaluation performed and on the method

used, and the process I have gone through to obtain the results of this research.

17/85


2 LITERATURE RESEARCH REVIEW ON IT EVALUATION

2.1 Introduction

The anticipation of new technologies and the way costs, benefits, and risks of Information

Technology (IT) are evaluated can make the difference between successful and unsuccessful

investments (Berghout & Renkema, 2005). It follows, therefore, that that profitable

investment in IT is not just a technical problem, but also has tactical and even strategic

aspects.

2.2 Objective

The aim of this literature research is to explore the theory regarding IT evaluation. With

knowledge of this the choice for an appropriate method can be made to perform the research

that is necessary for this thesis. The literature research will describe the evolution of IT

evaluation methods, the different methods that are available, and the problems and

deficiencies that occur when starting and conducting an IT evaluation.

The conclusion of this research will be the selection of an appropriate evaluation method that

can be used in the case study on the Virtual Study Place program at the Eindhoven University

of Technology (TU/e). Further on in this research the Virtual Study Place program will be

referred to as the notebook program.

The literature that was used for this research was partly recommended by the supervisor of the

University of Groningen, partly searched for in the library of Eindhoven University of

Technology, and partly found on the internet by means of search engines that are available

through the libraries of both universities. The main search expressions were “IT evaluation”

and “Information economics”. The literature that is chosen to be incorporated in this research

consists of some books and articles with an overview of the various evaluation methods. They

were chosen to obtain an idea of the different methods, and therefore to have the possibility to

choose a method for the evaluation of the notebook program. Other literature was used to get

information about earlier conducted research, and deepen the knowledge on the method that

was chosen. In the following table an overview of the literature is given, as well as the above

mentioned purposes for which the books and articles are studied.

18/85


Name article / book Writer(s) Year Type/use

1 Evaluating Information System Effectiveness Hamilton & Chervany 1981 Informative

2 Information Technology and Corporate Strategy:

A Research Perspective.

Bakos & Treacy 1986 Informative

3 Information Economics Parker, Benson &

Trainor

1988 Method

4 Information Strategy and Economics. Parker, Trainor &

Benson

1989 Method

5 Information Technology Evaluation: Is It

Different?

Powell 1992 Informative

6 It evaluation: Managing the catch 22 Willcocks 1992 Informative

7 Designing Local Evaluation Models for IT

Investments.

Kusters & Renkema 1994 Informative

8 Information Systems Investments: Evaluation at

the Feasibility Stage of Projects.

Willcocks & Lester 1996 Informative

9 Evaluation of Information System Proposals:

Design of a Decision Support Method.

Berghout 1997 Informative

10 Information as Competitive Advantage: the Role Powell & Dent- 1997 Informative

of Human, Business, and Technology Resources Micallef

11 Methodologies for information systems

investment evaluation at the proposal stage: A

comparative review

Renkema & Berghout 1997 Overview

12 Rethinking the Approaches to Information Serafeimidis & 1999 Overview

Systems Investment Evaluation.

Smithson

13 Information Technology Evaluation Methods &

Management

Grembergen 2001 Informative

14 Developing a frame of reference for ex ante IT/IS

investment evaluation.

Irani & Love 2002 Informative

15 Measuring information system service quality:

SERVQUAL from the other side.

Jiang, Klein & Carr 2002 Informative

16 From Business Strategy to IT Action Benson, Bugnitz &

Walton

2004 Method

17 Investment judgement of IT-projects Berghout & Renkema 2005 Overview

18 The Eleven Years of the European Conference on

IT Evaluation:

Berghout and Remenyi 2005 Overview

19 The enigma of evaluation: benefits, costs and risks Love, Irani, Standing, 2005 Informative

of IT

Lin & Burn

20 Information Technology, Strategic Decision

Making for Manager

Luca 2005 Informative

21 Investeringsbeoordeling- en management Renkema & Berghout 2005 Informative

22 The Advancement of IS Evaluation, a Literature

Review

2006 Overview

23 Operational Use evaluation of IT investments Al-Yaseen, Eldabi,

Lees & Paul

2006 Informative

24 Risk and Return of Information Technology

Initiatives

Dewan & Ren 2007 Informative

25 The Business Case for (ICT) Investment

Braaksma,

2008 Informative

Evaluation in Nonprofit Organizations.

Commandeur &

Berghout

19/85


2.3 Definitions

Evaluation is a broad concept; there are many different definitions of it in various scientific

and non-scientific literature. The “Van Dale Dictionary” defines evaluation as “the

determination of the worth of something”, and evaluation research as “research on the results

of taken measures”.

In the literature on IT evaluation some researchers use the general term for evaluation,

whereas some adapt it especially for the use in IT evaluation. Most of the definitions that are

used in the literature refer to a decision that has to be made whether to implement an

information technology appliance, or to decide which one to choose from several alternatives.

Both Berghout (1997) with the definition in his dissertation “Evaluation is defined of the

activity of comparing alternatives and the act of choice together” and Symons & Walsham

(1988) with their explanation: “The primary function [of IT evaluation] is to contribute to the

rationalization of decision making” focus on that purpose of IT evaluation. Other emphasize

the use of IT evaluation afterwards. That purpose is not always stated in the definition itself,

but when reference is made to the value of IT for the organization it can be interpreted as if

the evaluation is meant to be performed afterwards. There are many reasons why the

determination of the value of IT/IS systems in advance is so difficult that only afterwards

evaluation is performed in most cases: complexity, ambiguity, many interests involved, and

unfamiliarity are just a few (Berghout, 1997).

Willcocks & Lester (1996) write about “Establishing by quantitative and/or qualitative means

the worth of information systems to the organization”, whereas Eldabi et al. (2003) also

mention the time aspect as they define evaluation as: “A process that takes place at different

points in time, or continuously explicitly searching for (quantitatively or qualitatively) the

impact of IT projects”.

Remenyi & Sherwood-Smtith (1997) combine the two in their statement “Evaluation is a

series of activities incorporating understanding, measurement, and assessment. It is either a

conscious or tacit process which aims to establish the value or the contribution made by a

particular situation. It can also relate to the determination of the worth of an object”.

The most appropriate definition for an IT/IS evaluation would be the one of Eldabi et al.

(2003), because there is at least one advantage when an evaluation is performed at different

points in time. When an evaluation is performed during the implementation and use of the

IT/IS system, management has the opportunity to change the way the system is being used, or

even to adapt the system.

20/85


With that opportunity, it is possible to control the benefits for the organization. As soon as the

impact of an information system is clear, measures to try to enhance, or decrease, the impact

can be taken. Important is the frequency at which the evaluations take place, which shall be

different for various systems.

In the case of the notebook program there was only one evaluation after the implementation.

That evaluation was restricted to a survey regarding the experiences of the participating

students; there was no intention to evaluate the notebook program itself.

2.4 Evaluation methods

2.4.1 Introduction

Because of the greater impact of Information Technology and Information Systems within

organizations, and due to the greater amount of money that is involved in IT/IS investments,

researchers and practitioners emphasize the importance of IT evaluation. A well-performed

evaluation makes it easier to assign scarce resources to the right IT project, so that maximum

return on investment is achieved. An ex ante evaluation method will prevent misallocation of

funds. Another form, ex post evaluation, investigates the results of an information system

already in use.. A starting point for such an evaluation is the objectives as stated at the time of

the investment decision. Other additional advantages can be organizational learning, so that in

the future there will be a better decision-making process. And last but not least: by the use of

IT/IS the organization can gain competitive advantage, leaving behind competitors that fail to

use the right information systems.

In this section the evolution of IT evaluation methods will be described, as well as their

various categories. All methods do have their inadequacies: they will also be designated.

2.4.2 Evolution of IT evaluation methods

In the beginning of the use of information technology and systems the emphasis was on cost

saving by computerizing routine processes like easy to automate production activities or

routine activities in an accounting department like a department of finance The aim was to

streamline processes and reduce costs, in case of production facilities, i.e. through reduction

of waste of materials. In case of an accounting department there can be a cost saving because

the organization needs fewer employees to do the same amount of work. According to

Berghout & Renkema (2005) in that case there is hardly any investment issue, because the

costs for the information system are limited, and the benefits are easily visualized.

21/85


Evaluating such a situation can be done with a financial evaluation method, several of which

are described in the following part of this section (below).

On the other hand it is more difficult to assign benefits to IT when the applications are more

at the strategic level. Strategic information systems can aim to increase customer satisfaction:

the problem is the criteria which with these advantages could be measured.

A similar hard to measure advantage is the costs that are saved, or the benefits that are gained

regarding the loss or preservation of market share when implementing a strategic information

system. The last two issues are impossible to measure with only financial evaluation methods.

For that reason there is a shift towards the use of both financial and non financial methods, the

so called multi criteria evaluation methods. These multi criteria methods provide a more well-

balanced result of an IT evaluation. An overview of this is given in this table.

Characteristics “old” investment evaluation Characteristics “new” investment evaluation

Financial

Multi dimensional: financial and non

financial

Identification of cost savings

Emphasis on content

Local, stand alone facilities for a small

number of users

Functional oriented

Dominant role for the IT department

High technical risk systems

Optimization of project portfolio

Emphasis on evaluation in advance

Emphasis on project control: costs and

throughput time

Search for strategic opportunities

Integration of content and processes

Integrated infrastructural facilities with broad

availability

Beyond functional orientation

Dialog between managers, IT department,

and users

High financial and organizational risks

Avoiding poor solutions and uncertain

actions

Integration of evaluation in advance and

afterwards

Emphasis on project control: management of

benefits

Table 2.1 Evolution IT evaluation, source: Investment evaluation of IT projects. Berghout & Renkema, 2005.

22/85


2.4.3 Categories of IT evaluation methods

There are several researchers that make a distinction between various evaluation methods. In

this section the examination of classifications of a few of these methods is presented, without

trying to give an exhaustive overview.

Serafeimidis (1997) writes about technical streams, economic streams, and interpretive

alternatives. Each has its own purposes, subject of evaluation, time frames, methodologies,

and underlying assumptions. Powell (1992) refers to the dichotomization of methods into

objective and subjective techniques.

Objective measures try to quantify the system’s input and output in order to attach value.

Subjective methods rely on the opinions and attitudes of the users and builders of the

information system. Because of that reliance, subjective techniques can only be used ex post:

there is no applicability to evaluate an investment that has yet to be made.

Remenyi and Sherwood-Smith describe the same taxonomy with the concepts of summative

and formative, where summative is the same concept as objective and formative equals the

subjective methods as classified by Powell. Hamilton & Chervany (1981) disagree with this

statement; they speak of a goal-centered view and system-resource view which is similar to

the distinction drawn between summative and formative referred to above. The goal-centered

view is a way to assess the system’s effectiveness by determination of the task objectives of

the system, development of criterion measurements, and comparing the two. In contrast,

through the system-resource view, system effectiveness is determined by attainment of a

normative state. System effectiveness is measured by user job satisfaction.

Patel & Irani (2000) framed the different evaluation methods according to the following

taxonomy: economic, strategic, analytical and structured, and integrated. According to that

taxonomy they made a summary of appraisal techniques and approaches.

In their book Investment evaluation of IT projects (2005) Berghout and Renkema describe as

many as over seventy evaluation methods, divided in four main categories: financial, multi

criteria, ratio, and portfolio. In their paper “Methodologies for information systems

investment evaluation at the proposal stage: a comparative review” (1997) they describe a

comparison of methods that are classified according to these main categories. In the

comparison they name the following aspects: objects of the method, evaluation criteria,

support evaluation process, and type of outcome.

The comparison gives a fairly complete survey of the different categories and their use.

23/85


In the following part of this section the classification of Berghout & Renkema (2005) will be

the guideline. Financial, multi-criteria, ratio, and portfolio methods will be discussed in turn

The Financial Approach

Financial methods consider only effects of information systems when they are translatable in

money term (Berghout & Renkema, 2005). Zakierski (1987) refers to these methods as

objective ones which seek to quantify system inputs and outputs in order to attach value to

items (in Powell, 1992).

Payback period

This is the period between the moment that an investment is made, and the moment that the

total investment is recovered through net incoming cash flows. It depends on the organization

as to which time span is appropriate.

Average accounting rate of return

The first step is to estimate the financial return of an investment for each year of the projected

lifetime; this then is divided by the lifetime of the project. When that result is divided by the

initial investment sum the remaining ratio is the return on investment.

Net present value

This is a method that gives the opportunity to calculate the future costs and incoming cash

flows, and thus makes them comparable so that the right investment decision can be made.

Internal rate of return

The internal rate of return is the threshold at which the net present value equals zero. This

threshold is calculated after discounting the incoming and outgoing cash flows. If the

threshold exceeds the opportunity cost of capital, then the investment is worthwhile.

The Multi-Criteria Approach

Methods with a multi-criteria approach assess both qualitative as well as quantitative effects

of an investment project (Berhout & Renkema, 2005). Kusters & Renkema (1994) plead for a

local evaluation model in which financial criteria will always play a major part in the

decision-making process. However they also state that, given the positive and negative

indirect effects of an information system on the organization, the sole use of financial

evaluation will often result in an incomplete picture.

24/85


Information economics

In the field of evaluating IS investment proposals Parker et al.(1988, 1989) have given the

multi-criteria approach widespread publicity with their “Information Economics” method

(Renkema & Berghout, 1997).

The first criterion of the information economics method gives a financial evaluation of a

proposed IS investment, Parker et al. call this Enhanced Return on Investment. In this ROI

there is not only an emphasis on financial aspects; this method also provides additional

techniques to predict incoming cash flows. Next to this Enhanced ROI Parker et al.

distinguish between a business domain and a technology domain with regard to non financial

impacts and risks. The technology domain provides the business domain with the required IT

opportunities.

The business domain pays for the used resources and focuses on optimal deployment of the

given IT opportunities. As such the total evaluation of an IT proposal takes place in three

steps, having focuses on financial, business, and technology dimensions.

New Information Economics

This method has evolved from the Information Economics that was developed in 1988. The

NIE Method (Benson et al., 2004) consists of five NIE practices: strategic demand/supply

planning; innovation; prioritization; alignment and performance management. Like

Information Economics its main goal is to let IT fundamentally improve how the business

performs. To do so, business management must be directly involved in IT decision making.

The generic process model incorporates NIE practices and supporting practices such as

portfolio management to provide a more integrated and effective strategy-to-bottom-line

connection. These practices close the gaps between strategy-to-results. The practices, when

integrated with existing organization’s management processes such as budgeting and annual

planning, will give the organization a sound set of management processes that satisfy the goal

of translating business strategy into IT goals that produce the right business results. The

practices used in a connected set of management processes will enable management to control

IT spending and improve IT’s impact on the mission performance.

Strategic Investment Evaluation and Selection Tool Amsterdam (SIESTA)

Developed at Amsterdam University, this method is one of the most extended methods. It

consists of questionnaires and an application is developed, so that the interest of a given

criterion can be well determined. The decision criteria are separated from the business and

technology domain, as well from the different levels of decision making.

25/85


Balanced Score Card (BSC)

The BSC is a frequently used method for IT evaluation. This method is a supplement to a

financial approach and distinguishes a financial, customer, internal business, and an

innovation and learning perspective. Much research regarding the use of the BSC usage in IT

evaluation has been conducted by Van Grembergen. In his book “Information Technology

Evaluation Methods & Management (2001) he describes research by himself and Saull

regarding the linking of the IT BSC with the business BSC, as well as contributions provided

by Meyerson, Verleun & Berghout & Looijen, and Eickelmann. With this Van Grembergen

emphasizes the growing usage of, and interest in, BSC in IT evaluation.

The Ratio Approach

Due to the demand for benchmarking financial data, research has been conducted into the

usability of, in particular, financial ratios. Ratios do not only refer to financial data; they can

also be related to the numbers of employees, or to a production unit.

Return on Management (ROM)

The main assumption with this method is that the management style of an organization

influences the benefits of Information Technology. In this method the added value of

management is related to the costs of that same management. To calculate the ROM there has

to be access to the (not public) MPIT database. This database contains data of approximately

300 organizations over several years, and can be used to screen the organization as a whole as

well as evaluate an individual decision.

IT-Assessment method

IT-Assessment is another method that uses benchmarking as a means to evaluate information

facilities. Like the database used to calculate ROM, these benchmarking data are not public.

An important part of this method is the formulation of financial and non financial ratios.

These ratios are compared with those of other organizations which act as the benchmark.

These ratios can also be used to make a historical analysis of the organization’s investments.

26/85


The Portfolio Approach

Portfolios are frequently used methods to support decision making processes. A well known

method is the Growth Share Matrix developed by the Boston Consulting Group. In the

evaluation of IT projects the projects are placed in the matrix with regard to several evaluation

criteria.

Bedells’s method

With this method three questions will be answered: is it necessary for the organization to

invest in IT; which organizational activities should be computerized; and which IT

applications should be developed? Three stakeholders are asked to answer those questions:

senior management, user management, and IT specialists. The main issue of this method is

the balance between quality and importance. An information system is important if it supports

important activities, and if those activities are important to the organization. With all these

data the various IT projects can be prioritized.

Investment portfolio method

The investment portfolio evaluates IT investment proposals simultaneously on three criteria:

contribution to business domain, contribution to technology domain, and the financial

consequences by means of net present values calculation.

To get the right proposition for the investment decision three different stakeholders are

involved: senior management, IT management, and the project management of the

development project. They evaluate the investment proposal on one of the three evaluation

criteria as mentioned above.

Investment mapping

Renkema & Berghout (1997) also mention investment mapping as a possible evaluation

method. The developer (Bedell, 1985) of investment mapping has intended it to serve as a

method of relating IT investment to organizational and business needs. In a way the purpose

is the same: to come to the right decision about which IT investments proposals to go with.

The method uses a map marked with the dimensions of benefits, ranging from more tangible

issues like improving the production, to less tangible issues like increasing business. The

other dimensions are the investment orientations: infrastructure, business process, and market

influence. When IT investment proposals are located on the investment map, gaps and

overlaps will be revealed.

27/85


2.5 Problems and deficiencies in IT Evaluation

There probably is no field of study where the methods of development are without problems

or deficiencies, and the discipline of IT evaluation is no different. And as in various fields of

study, the saying goes: “You cannot fully understand a subject until it is measured”. The mere

fact that there is no general IT investment evaluation method available that covers all possible

questions and circumstances indicates that it is almost impossible to develop such a method.

But what makes it so difficult to evaluation IT investments? Berghout & Renkema (2005)

describe four aspects that have an influence in the evaluation: the benefits of IT are hard to

estimate, measure, and control; the costs are high and unpredictable; there are great

uncertainties and considerable risks; and there are communication problems and conflicts of

interest between the various stakeholders. Several researchers speak of the difficulties they

encounter when trying to decide what criteria and measurements to use.

There are various causes that make evaluation so hard to accomplish: the costs and benefits of

IT are integrated in the total costs and benefits of the organization: it is therefore extremely

challenging to extract the exact part of IT from those figures.

Besides the integration of the tangible benefits, there is a problem with the measurement of

intangibles, which are most often the key to many investment decisions (Lucas & Weill,

1993). It is almost impossible to measure things like job satisfaction, because not only the

personal perspective of the concept is different among people, but the values they have about

satisfaction also can vary. That makes it difficult to produce a reliable analysis of the research.

Other problems in evaluating are: inadequately defined objectives and measurements; the fact

that objectives and measurements used in the evaluation may differ from those defined

initially; and the issue that individual perception may differ on what the objectives and

measurements are (Hamilton & Chervany, 1981). Mirani & Lederer (1998) and Khalifa et al.

(2001) state that because IT evaluation is contextual and subjective there is no unified “one-

size fits all” concept possible. Kusters & Renkema (1994) argue for a model that reflects local

organizational circumstances and language that is in use in an organization. In that

perspective Braaksma et al. (2006) have developed the Public Value Framework, because the

drive of for-profit organizations (shareholder value) is not applicable. In a non for-profit

organization there is the need to create value for a diversity of stakeholders.

28/85


And if all these problems are still not enough, an evaluation can be influenced by ethical

issues such as the distribution and exercise of power, as well as the culture that is present in

the organization (Ballantine et al., 2000).

2.6 Conclusion

The presence of information technology and information systems in organizations today is

significant. Most of the IT/IS are embedded in the organization in such a way that the

processes of the organization can not be performed without them, or only at higher costs.

Besides this significance, investments in IT/IS also represent a substantial part of the total

investments of an organization. Both aspects should be a reason to evaluate IT investments,

both ex ante and ex post. The former is to prevent available resources being ascribed to

projects with the least benefits, or worse; with a negative impact on the organization. The

latter is to give an indication of the benefits, and to help improve the decision-making process

and organizational learning. In spite of all these advantages, not many organizations have IT

evaluation embedded in their processes. The reason for the reluctance towards IT evaluation

lies in the problems that come with it. It is difficult to specify the right criteria.

It is also not always clear what the objectives initially were. If the objectives were clear then

there can be differences in perspective between the various stakeholders. A final reason is

that political issues may obstruct evaluation.

In view of all these problems it seems impossible to develop one generic model for IT

evaluation, especially when the organization’s context is to be considered. This means that for

every IT evaluation, the evaluator and his team have to combine several techniques,

depending on the objective of the evaluation.

2.7 Selection of an evaluation method for the notebook program

For the evaluation of the notebook program of the Eindhoven University of Technology the

method New Information Economics (NIE) is chosen. This method seems, of all the methods

that are included in the literature research, the best to perform the evaluation of the notebook

program. The method is recently developed, on the basis of the Information Economics

method of 1989. This means that the concepts of a proven method are integrated in one that

meets the demands of the present time.

This method is chosen because it emphasizes the importance of alignment of IT with the

strategic intentions of the organization.

29/85


In an evaluation like the one on the notebook program, where strategic objectives are the main

reason to implement IT facilities, this alignment is the far most important issue to evaluate.

Next to the Alignment Practice, as it is being called in NIE, a Performance Measurement

Practice is also presented, which is divided in financial and other performance measurements.

NIE consists of more practices: Innovation, Prioritization, and Strategic Demand/Supply

Planning Practices, which makes it suitable for ex ante evaluations. But the practices are well

suited to perform separately, which makes the method fit to use for an ex post evaluation.

The NIE Method consists of five NIE practices: strategic demand/supply planning;

innovation; prioritization; alignment and performance management, and its main goal is to let

IT fundamentally improve how the business performs at the bottom line. The practices, when

integrated with existing organization’s management processes such as budgeting and annual

planning, will give the organization a sound set of management processes that satisfy the goal

of translating business strategy into IT goals that produce the right business results.

This is visualized in the figure below, which Benson et al. (2004) in their book “From

Business Strategy to IT Action” call the Strategy-to-Bottom-Line Value Chain. The five

practices of NIE and supporting practices are embedded.

Fig. 2.1 Strategy-to-Bottom-Line Value Chain (Benson et al., 2004)

For the evaluation of the notebook program not all the practices are applicable. Because the

evaluation has an ex post character the Strategic Demand/Supply Panning has already taken

place.

30/85


At the time the decision was made to implement the notebook program, the Executive Board

and the Departments came to a consensus with the IT managers on what IT could do to help

achieve the intended strategy. As far as Innovation is concerned: the strategy needed IT to

fulfill it, it was not IT that offered a new strategy at that time. There was no new business

opportunities IT showed at the business, just another way to fulfill the strategy’s needs.

Because this evaluation is done after the decision in favour of the notebook program is taken

and the program already is implemented, the practice Prioritization is not necessary. That

leaves the practices Alignment and Performance management. These two are suitable to

perform an ex post evaluation like the one on the Eindhoven University of Technology

notebook program. The evaluation is presented in the next chapter.

31/85


3 EVALUATING THE NOTEBOOK PROGRAM TU/e

3.1 Introduction

In this chapter the chosen evaluation method New Information Economics is applied to the

notebook program of Eindhoven University of Technology. Before an evaluation method

could be chosen, research on the various available methods was performed. The results of that

research are presented in chapter 2, as well as the argumentation of the choice in favour of

NIE. For the evaluation of the notebook program all the practices of NIE are not applicable,

the practices Alignment and Performance management are chosen to conduct the evaluation

with. The argumentation to use just these two practices is also presented in chapter 2.

To perform the evaluation on the notebook program of the TU/e the first thing that has to be

done is to determine the objectives of the notebook program. As the notebook program is

already in existence the evaluation will have an ex post approach; the overall objective will be

the investigation of the results of the program. A starting point for such an evaluation is the

objectives as stated at the time of the investment decision. A possible bottleneck as described

by many researchers in evaluation is changing objectives, it is important to conduct research

on the possible changes of the objectives of the notebook program.

3.2 Strategic intentions

Before the practices can be applied, and the evaluation can be performed, the strategic

intentions of the notebook program have to be clear. In the table below these strategic

intentions are summarized. The strategic intentions are withdrawn from the mission of the

Eindhoven University of Technology as stated in the strategic statement in 1997, taking in

consideration the assignment to fulfill the strategic statement in a feasible way.

Strategic intention Description Goal Key metric

A.

Integrated IT facilities for

all students and

employees in a feasible

way

B.

Embedding IT in the

education program

C.

Acquire competitive

advantage compared to

other universities in the

Netherlands

The possible use of IT

facilities for all students

and employees, in a way

that the costs of those

facilities are manageable

Increasing the use of IT

facilities in the education

program. For this

research there is a

delimitation on mail

functionality

Acquire competitive

advantage compared to

other universities in the

Netherlands by means of

the notebook program

The possibility to use IT

facilities at the user’s

convenience.

Manageable costs for the

university

Equip the students with

the necessary skills to use

IT resources for their

future positions

Table 3.1 Strategic intentions of the notebook program

Costs of the various ways

of implementing the

strategic intention

The increase of the use of

mail between students

and teachers

To attract more students Number of first-year

students in the past

Number of first-year

students of the various

universities

32/85


Above strategic intentions can also be visualized in the conceptual model below, which

implies the question whether the notebook program is of influence on costs, changes is

communication between teacher and student, and competitive advantage.

3.3 The Alignment practice

Competitive advantage TU/e

Changes in communication between

student and teacher

Financial aspects

for students & university

Fig. 3.1 Strategic intentions notebook program

The Alignment practice looks at the existing activities and relates them to the business

strategic intentions. With that it provides a way to review decisions about investments in the

past, taking in account present and future needs. If necessary it gives way to free resources for

activities that have got a higher impact on the wanted strategic intentions. The practice is

divided in three parts: the first is Strategic Alignment and is about addressing the alignment of

asset pools (applications, services, infrastructure, and management) to the business strategic

intentions. The second part is addressing how well the asset pools are aligned to each other.

This part is called Internal IT Alignment. The third one is called Functional Alignment and is

concerned with the service level, quality, functionality, technology, and intensity of use of

each asset pool. All the parts of the Alignment practice answer the same question: “Does the

existing IT activities promote or inhibit the strategic intentions and operational

requirements?”

33/85


3.3.1 Part I Strategic Alignment

This part of the Alignment practice is about addressing the asset pools, and about how well

they support the strategic intentions and the operational requirements. At first it is necessary

to name the elements of the asset pools. These are summed up in the table below. Because the

elements are not only activities of one supportive bureau, the names of bureaus involved are

also mentioned.

I Service Part of organization TU/e

1. Enrollment first-year students Student administration

2. Distribute information notebook program Student administration

3. Advice choice about notebook Departments

4. Make choice notebook first-year students Executive Board

5. Purchase notebooks Student administration

6. Compose image with necessary applications Departments & IT Services

7. Helpdesk / on-site support IT Services & Departments

II Applications Part of organization TU/e

1. Student Information System Student administration

2. Studyweb (electronic learning system) Departments

3. Website provisioning applications IT Services

4. Notebook Information System IT Services

III Infrastructure Part of organization TU/e

1. Network LAN & Wireless IT Services

2. Software licenses IT Services

3. E-mail and storage access IT Services

4. Internet IT Services

5. Security IT Services

IV Management Part of organization TU/e

1. Budget management Student administration

2. SLA partner management Student administration

3. Insurance management Student administration

4. Compose images planning IT Services

5. Operational support management IT Services

Table 3.2 Asset pools Strategic Alignment

The purpose of Strategic Alignment is to examine the alignment of the asset pools with the

strategic intentions and the operational requirements.

34/85


That purpose is visualized in the figure below.

Services

Stategic

Intentions

Applications

Operational

Requirements

(Business

Processes)

Fig. 3.2 Strategic Alignment, source Benson (2004)

Infrastructure

It is necessary to assess the various elements of the asset pools in their relationship to the

three strategic intentions. For that an alignment the alignment scale shown below is

determined.

3 Critical enabler Without this service, the strategic intent cannot be

established

2 Enabler Without this service the strategic intent can be

accomplished, but at some additional costs

1 Indirect enabler Service is indirectly related to achieving the strategic intent

0 No effect Service has no effect on achieving the strategic intent

Blank NA Not Applicable, service is not used

-1 Minor inhibitor Service is indirect related to inhibiting the achievement of

the strategic intent

-2 Inhibitor With this service the strategic intent can be accomplished,

but at additional costs

-3 Critical inhibitor With this service, the strategic intent cannot be

accomplished

Table 3.3 Alignment Assessment Scale

35/85


The assessment of the varying elements of the asset pools is done within the interviews with

the people of Student administration, IT Services, and the teachers and researchers that were

interviewed.

I Service Strategic Strategic Strategic

intention A intention B intention C

1. Enrollment first-year students 3 0 2

2. Distribution information notebook program 2 2 3

3. Advice choice about notebook 2 0 2

4. Make choice notebook first-year students 2 0 3

5. Purchase notebooks 2 2 3

6. Compose image with necessary applications 2 3 2

7. Helpdesk / on-site support 3 3 3

II Applications Strategic Strategic Strategic

intention A intention B intention C

1. Student Information System 1 1 0

2. Studyweb (electronic learning system) 3 3 2

3. Website provisioning applications 3 3 2

4. Notebook Information System 2 1 0

III Infrastructure Strategic Strategic Strategic

intention A intention B intention C

1. Network LAN & Wireless 3 3 2

2. Software licenses 3 3 2

3. E-mail and storage access 3 3 2

4. Internet 3 3 2

5. Security 2 2 1

IV Management Strategic Strategic Strategic

intention A intention B intention C

1. Budget management 2 1 0

2. SLA partner management 2 3 0

3. Insurance management 2 2 0

4. Compose images planning 2 1 0

5. Operational support management 3 2 0

Table 3.4 Results Strategic Alignment Asset pools

The above table shows that all the elements in the asset pools are aligned with strategic

intention A. (Integrated IT facilities for all students), although the feasibility is not part of the

assessment. Also the alignment with strategic intention B. (Embedding IT in the education

program) is clear to see through the scales that the assessment has resulted. To gain

competitive advantage (Strategic intention C.), the table shows that the infrastructure, the

choice of the notebook, and the information to prospective students is important, but that the

management of the processes is not of influence on the strategic intent.

36/85


3.3.2 Part II Internal IT Alignment

The assessment of the internal alignment is the one of the alignment between applications and

infrastructure. Is the infrastructure supporting the applications? These relations are visualized

in the figure below.

Applications

IT Management

Services

Services Infrastructure

Fig.3.3 Internal Alignment, source Benson (2004)

The same alignment scale as used for the Strategic Alignment is used for this assessment.

III Infrastructure Application Application Application Application

1

2

3

4

1. Network LAN & Wireless 3 3 3 3

2. Software licenses 1 3 3 1

3. E-mail and storage access 2 3 1 2

4. Internet 3 3 3 0

5. Security 3 3 3 2

Table 3.5 Internal alignment infrastructure

The table shows that all infrastructural elements are in alignment with the used applications.

The score on email in particularity is important because the mail application that is used at

Eindhoven University of Technology is Microsoft Exchange, and application 2 (Studyweb) is

build on functionality of that product.

Note: In this research the third part of the Alignment Practice, Functional Alignment is not

performed because the evaluation is about the notebook program as a mean to achieve the

strategic intentions mentioned earlier.

37/85


The notebook program consists of more than one application; it also contains the changes in

communication and the possible competitive advantage. The functional alignment assessment

of this practice seems not to be of use in this context. The necessary functional evaluation is

conducted as part of the Performance measurement Practice and described in section 3.4.2.

3.4 Performance Measurement Practice

The Performance Measurement Practice, like the Alignment Practice, consists of three

dimensions: Cost Performance, Quality and Service Levels, and Process Measurement.

These dimensions and their interconnection are visualized in the graph below. The

relationship between the Service and Quality Level, and Costs are clear. Less clear is the

influence that the quality of processes has on the relationship between costs and quality. For

instance: an improvement in the costs of purchases can occur if the process “Purchasing” is

well defined and fine-tuned.

Process Development

3.4.1 Part I Cost Performance

Cost Performance

Quality and Service Level

Fig. 3.4 Performance Dimensions, source Benson (2004)

Although cost reduction was not one of the primary goals of the notebook program, one of the

questions asked when the Executive Board gave the assignment to develop a way to fulfill the

strategic intention “Integrated IT facilities for all students and employees” was to do so in a

feasible way. That meant: against the lowest possible costs, without putting the main goal

(integrated IT) at stake.

38/85


With that basic assumption in mind in this research costs and cost reduction are being

weighed for just that parts of the IT investments that vary between the alternative ways to

fulfill the strategic intentions. There is no difference in the investments in technical

infrastructure, applications and so on whether the end-point of the integrated IT facilities is a

notebook or a desktop PC.

A difference does exists between the costs of the PC’s or notebooks that are used to fulfill

Strategic Intention A, the prizes of the devices are different, especially if the technical

specifications have to be as equal as possible. Another possible difference could occur in the

costs of service and support of these end-points of the integrated IT facilities.

In de following table these two alternatives (desktop or notebook) regarding the financial

aspects are presented. In the table only the costs that are paid by the university are presented.

In case of the notebook alternative the student also pays a part of the total sum (except for the

housing part), this explains the relatively low costs for the notebook alternative, whereas in

the desktop alternative all costs have to be paid for by the university

All costs per year in € Desktop(including

accessories)

Notebook(including

accessories)

Purchase value 350 200

Support services 250 150

Hardware service 0 50

Housing 460 345

Total costs 1.060 745

Table 3.6 Costs of alternative configurations

Some additional remarks have to be made regarding the figures in the table above:

• These figures are an indication;

• The notebook alternative also includes insurance, and the use of a temporary spare

notebook in case of a broken notebook;

• The extra amount for hardware services has to be counted for because the guarantee

that is purchased with the notebooks is on basis of carry-in, whereas the student

notebooks are being repaired on-site (on the university grounds).

Next to that the guarantee is only contracted for only the first three years, whereas the

actual service at Eindhoven University of Technology is extended to five years.

39/85


That means that spare parts and labour costs of the service provider in the last two

years have to be paid separately.

This topic of on-site handling and extended guarantee is not applicable to desktops,

because they are purchased with an on-site guarantee contract, and are being replaced

after three years;

• The costs of Support services of notebooks is considerable lower, two causes are

responsible for that difference.

o The services that are to be delivered to notebooks is based on carry-in, the

service desks are located in every departmental building, and in the building of

the central student facilities. This means that students come to the service desk

with their notebook and problems. When the desktop alternative was

implemented, the employees that support the computer rooms had to go to the

rooms that are scattered across the buildings.

o The knowledge students acquire of the IT facilities that are offered by the

notebook program. This is one of the Strategic Intentions, but also has a

positive influence on the costs of support services. Students are the system

engineers of their own notebook, and because of the knowledge they acquire

whilst operating and managing the notebook they simply generate less service

calls.

Both causes taken into consideration, all the interviewed departmental IT coordinators

mentioned that they would have at least one fte support staff more to solve problems

with desktop PC’s in computer rooms. Next to that; less support staff is necessary in

the buildings that are not part of a department, like the public lecture building. The

support staff in those buildings is reduced with at least 4 fte. These reductions in

personnel costs are taking into account when calculating the costs for Support services

of the notebook alternative;

• One of the most expensive parts of both alternatives is also presented in the table: the

housing costs. If, like mentioned before, a room has to be equipped with desktop

computers; it can not be used for something else. Next to the singular character of

these rooms, there is another practical disadvantage to using desktops: the simply

require more space. If a student needs to work with a desktop computer he needs at

least 2 m 2 , with a notebook 1,5 m 2 is usually sufficient.

Of course this problem can be solved with some models of desktops and flat screens,

but they make the configuration more expensive. Another argument why notebooks

40/85


need less space for housing is that almost any place is useful, as long as the needed

facilities like the wireless network are available. This means that students can work,

alone or in groups, in the university restaurant, in lecture rooms, or anywhere they

choose without the university providing them with special places.

The above mentioned cost reductions are benefits for the university itself.

In the academic year 2007-2008 the student had to pay € 900,00. The rest of the costs of the

notebook and al the above mentioned services are subsidized by the university in the form of

an obligation contract the student signs. After three years of study at Eindhoven University of

Technology the obligation is finished.

The university converts the obligation into a gift, and the student acquires the ownership of

the notebook. Although the contract only lasts for three years, the students that make use of

the notebook program receive hardware support for five years and software support for six

and a half years. This extended service means that the university’s contribution is higher than

that of the student.

The costs for the students can be paid in once, but students can also make use of a loan of

which the university pays the interest.

The main reasons that students make use of the notebook program are the high technical

specifications of the notebook, and the services provided. And although sometimes a mere

notebook can be bought for less, 95% of all first-year students to make use of the notebook

program.

Another cost advantage of the use of notebooks in the education program emerged while

talking to some of the researchers and teachers. Two of them, and several IT coordinators,

mentioned a change in the use of sophisticated experimental systems in different laboratories.

In the past there were just a few of those systems available for students, but due to the

introduction of the notebooks researchers developed peripheral equipment to use with the

notebook. This means that the during the practices in the laboratories students could use the

power and screen of the notebook in stead of the expensive, and therefore few available, stand

alone equipment.

None of the interviewees could give an indication of the cost reduction that development

caused, but they did say that the use of the notebook in this way was seen by both teachers

and students as a big improvement due to the notebook program.

41/85


3.4.2 Part II Quality and Service levels

With Quality and Service Levels measures it is possible to assess, next to the alignment to the

strategic intentions in the Alignment Practice, the Quality and the Service Levels of the

various element of the asset pools. With the total of assessment management can decide if the

resources that were assigned at the different elements of the asset pools in the future have to

be the same, less, or even higher.

Although, concerning the notebook program, it is not clear what the resources were in the past

because of the time that elapsed between the first implementation and the present time of

evaluation, it is useful to perform this exercise to see which elements are important to achieve

the strategic intentions and how they are performing in both Service Level and Quality. It

gives the management the opportunity to decide if more resources have to be assigned to a

particular entry. The marks that are assigned for quality and service level can vary between 0

and 3. The marks on availability of the infrastructure and applications are assigned on basis of

various reports regarding uptime of the entries. Although the reports showed that the technical

availability was to be rated with a 3, the actual period of use of the entries was taken into

account. For instance: the enrollment of first-year students is a service that is not performed

24 hours a day, for 7 days a week. But Studyweb and the Wireless network are available

during that complete period. The responsiveness is rated on the way the entry reacted when a

call was made to it.

I Service Elements of Service level Score

1. Enrollment first-year students Availability

2

Responsiveness

3

2. Distribution information notebook program Availability

2

Responsiveness

3

3. Advice choice about notebook Availability

1

Responsiveness

3

4. Make choice notebook first-year students Availability

1

Responsiveness

3

5. Purchase notebooks Availability

1

Responsiveness

3

6. Compose image with necessary applications Availability

2

Responsiveness

3

7. Helpdesk / on-site support Availability

2

Responsiveness

3

II Applications Elements of Service level Score

1. Student Information System Availability

3

Responsiveness

3

2. Studyweb (electronic learning system) Availability

3

Responsiveness

3

3. Website provisioning applications Availability

3

Responsiveness

3

4. Notebook Information System Availability

2

Responsiveness

3

42/85


III Infrastructure Elements of Service level Score

1. Network LAN & Wireless Availability

3

Responsiveness

3

2. Software licenses Availability

3

Responsiveness

3

3. E-mail and storage access Availability

3

Responsiveness

3

4. Internet Availability

3

Responsiveness

3

5. Security Availability

3

Responsiveness

3

Table 3.7 Service Level Asset pools

I Service Elements of Quality Score

1. Enrollment first-year students Reliability of process

3

Reliability of result

3

2. Distribution information notebook program Reliability of process

3

Reliability of result

3

3. Advice choice about notebook Reliability of process

3

Reliability of result

3

4. Make choice notebook first-year students Reliability of process

3

Reliability of result

2,8

5. Purchase notebooks Reliability of process

3

Reliability of result

3

6. Compose image with necessary applications Reliability of process

3

Reliability of result

3

7. Helpdesk / on-site support Reliability of process

3

Reliability of result

2,5

II Applications Elements of Quality Score

1. Student Information System Functionality

2,8

Accuracy of application & data

3

2. Studyweb (electronic learning system) Functionality

2,5

Accuracy of application & data

3

3. Website provisioning applications Functionality

2

Accuracy of application & data

3

4. Notebook Information System Functionality

2,8

Accuracy of application & data

3

III Infrastructure Elements Quality Score

1. Network LAN & Wireless Functionality

3

Reliability

3

2. Software licenses Functionality

3

Reliability

3

3. E-mail and storage access Functionality

3

Reliability

3

4. Internet Functionality

3

Reliability

3

5. Security Functionality

3

Reliability

3

Table 3.8 Quality Asset pools

43/85


3.4.3 Part III Process Measurement

“Process measurement (and management) is a way to affect the IT organization’s

performance drivers. Measuring and managing processes is about measuring and managing

the causes behind costs and service-level results” (Benson, 2004). Regarding the notebook

program several processes are important to make sure that the costs for both students and

university stay at the lowest possible level, without making any concessions to the quality or

service level of the offered services. Although a process model for the fine-tuning or

monitoring of the processes that make up the notebook program is never used, the processes

were improved significantly since the start of the program. Not alone did this for instance lead

to faster decision making on the notebook choice and the applications to be used, but also to

the extension of facilities like the possibility for students to place an image on their notebooks

without interference of a service desk, which at itself lead to a cost reduction regarding

personnel.

3.5 Effects on communication

One of the initial goals of the notebook program was to provide students with IT means and

knowledge about those means by using Information Technology in the education program.

The matching strategic intention is “Embedding IT in the education program”.

Due to the ways to communicate with the notebook, and the provided applications, the way

students communicate has changed over the years. The notebook improved the possibility to

communicate enormously, with the use of a notebook it is possible for students to e-mail their

teacher (or colleague students) at any time or place. And although some students and teachers

find that behavior rather disturbing regarding their privacy, most students think it is a

tremendous increase in flexibility that they feel is convenient.

In spite of the fact that it is almost impossible to determine the real influence of the notebooks

regarding the aspects of communication any time and any place, some earlier inquiries among

students concerning the notebook program indicate that the use of IT in communication has

increased considerably during the past years. A questionnaire sent to all notebook users three

years after the introduction of the notebook program reveals that 45% of the notebook users at

that time used the notebook to communicate in education on a regular basis, most of them

even frequently. For private communication usage this percentage was even higher; 70% of

the notebook owners used the notebook for communicative purposes. 87% of the

communication was carried out by means of e-mail. Only 18% of the students frequently used

the notebook to chat.

44/85


Looking at the degree of students that communicated electronically at that time with teachers,

there is a much lower percentage (a minority of only 17%) that communicated via e-mail with

teachers on a regular base (Notebooks and students, 2000).

80

70

60

50

40

30

20

10

0

Percentage communication by notebook

70 68

17 16

Colleagues Friends Teachers Unknown

Fig. 3.5 Graph of communication by notebook, source Notebooks and students (2000)

To determine whether an increase in communication occurred since then, a few questions

regarding the use of electronic communication, i.e. e-mail were asked to a group of students.

The questions not only refer to the use of notebook in communication, but also to the use for

general study. Also a few questions were asked about the availability of IT facilities, and the

role students think the notebook will be in the future.

The survey was conducted by email and by interviews, 180 students were asked to fill in a

form with the questions. To get a proper reflection of the TU/e student population the

interviews were conducted in various departmental buildings. The emails with the appeal to

fill in the form were sent to students of different study generations, reflected by the year the

student enrolled in his educational program. Because of the use of interviews, the response

was fairly high: 85 students answered the questions. The questions and the summarized

answers are presented in appendix C. The analysis of the data is presented in section 3.7.3.

Next to the questions to students, also interviews with some teachers and IT coordinators of

the various departments were conducted. They also, like the students, stated that in their

opinion communication by electronic means would not have expanded in such a short period

of time without the implementation of the notebook program. They thought that it was not

only attributable to the notebook itself, but also the organization around the use of a notebook

in the education programs. Students for instance are given an email account from the moment

they enroll in an education program.

45/85


That means that they can reach their teachers and other students from the beginning. It also

means that a teacher, or for example the student administration, can contact the student

without any problem.

3.6 Competitive advantage

Because the gaining of competitive advantage was also a strategic intention of the notebook

program, it is necessary to conduct research on the influence of the program on the decision

of prospective students concerning the university to choose for their education.

At first, a comparison was made of the number of first-year students at Eindhoven University

of Technology itself. The start of the comparison is 1997, the year of the introduction of the

notebook program, until 2006. The data of the comparison is extracted from the Statistical

Abstract and the website of the Education and Students Service Center.

The number of first-year students is shown in the table below and visualized in the graph.

Eindhoven University of Technology

First-year Difference Difference

Year students (numbers) (%)

1997 1074

1998 1217 143 13.31

1999 1420 203 16.68

2000 1419 -1 -0.07

2001 1431 12 0.85

2002 1467 36 2.52

2003 1417 -50 -3.41

2004 1465 48 3.39

2005 1410 -55 -3.75

2006 1423 13 0.92

Table 3.9 student number Eindhoven

1600

1400

1200

1000

800

600

400

200

0

First year students Eindhoven

1997 1998 1999 2000 2001 2002 2003 2004 2005 2006

Fig. 3.6 Graph student number Eindhoven

The increase of the number of students at Eindhoven University of Technology in the first

two years after the introduction of the notebook program is significant in comparison with the

following years. To place a legitimate claim on the positive influence of the notebook

program on the number of first-year students, it is necessary to compare them with other

universities.

46/85


In this case it seems logical to compare with the other two technical universities in the

Netherlands; it seems rather unlikely that a student who has an interest in a general education

will enroll in a technical university just to obtain a notebook with favourable benefits.

In the following tables and graphs, the number of first-year students is presented at Delft

University of Technology and the University of Twente over the same period of time.

Delft University of Technology

First-year Difference Difference

Year students (numbers) (%)

1997 2345

1998 2434 89 3.80

1999 2434 0 0.00

2000 2316 -118 -4.85

2001 2297 -19 -0.82

2002 2095 -202 -8.79

2003 2380 285 13.60

2004 2479 99 4.16

2005 2599 120 4.84

2006 2703 104 4.00

Table 3.10 student number Delft

University of Twente

Year

Firstyear

students

Differenc

e

(numbers)

Differenc

e (%)

1997 1056

1998 1094 38 3.60

1999 1148 54 4.94

2000 1204 56 4.88

2001 1225 21 1.74

2002 1390 165 13.47

2003 1481 91 6.55

2004 1481 0 0.00

2005 1604 123 8.31

2006 1643 39 2.43

Table 3.11 student number Twente

3000

2500

2000

1500

1000

500

0

1800

1600

1400

1200

1000

800

600

400

200

0

First year students Delft

1997 1998 1999 2000 2001 2002 2003 2004 2005 2006

Fig. 3.7 Graph student number Delft

First year studentsTwente

1997 1998 1999 2000 2001 2002 2003 2004 2005 2006

Fig. 3.8 Graph student number Twente

47/85


Both universities have an increase in first-year students in one year, which is not explicable

by, for instance the start of a notebook program, as seems to be the case in Eindhoven. Delft

and Twente do have a kind of a notebook program, but those are in many ways incomparable

to the one in Eindhoven. Moreover, the year in which the increase occurred is not the year

after which they initiated a notebook program.

Twente started with a notebook program in 2004, whereas the significant increase of first-year

students occurred in 2002, so the notebook program could not have been of influence on the

increase. In Delft there an increase occurred in 2003, which was also before the introduction

of notebooks there.

Although the increase of first-year students at Eindhoven University of Technology for a few

years after the notebook implementation is significant, the results of the research on the

influence of the notebook program show no sustainable competitive advantage for any of the

universities. A competitive advantage is called sustainable if it cannot be copied, replaced

with a substitute, or diminished by activities of competitors. Nor is it sustainable when the

product or service becomes unneeded due to changes in the environment (Porter, 1980).

The conditions that would make the advantage sustainable are not applicable for the program.

It is possible to copy it, although the notebook program of Eindhoven University of

Technology has not yet been copied by one of the competitors. The main strategic intention of

the notebook program, to provide the students with the necessary skills, can also be achieved

by other means, for instance by providing the students with sufficient personal computers in

computer rooms. The final condition besides the environmental one: diminishing activities of

competitors can occur. If these competitors were to implement a notebook program that is

comparable, or another one that attracts more students, it may be that prospective students

will enroll in their education programs. As far as the environmental condition are concerned:

since the implementation of the notebook program the world has changed, more and more

students already have some kind of computer at their disposal, a PC or notebook.

Still, the number of first-year students that make use of the notebook program at Eindhoven

University of Technology is high: almost 95%. But that does not make it a competitive

advantage!

48/85


3.7 Analysis

3.7.1 Strategic Alignment

The strategic intentions that were extracted from the Strategic Blueprint of Eindhoven

University of Technology were the subject of the Alignment Practice of the New Information

Economics method. The method consists of three parts, of which two were used for the

evaluation of the notebook program. The first part, the strategic alignment shows that the

activities that are being performed are in alignment with the strategic intentions. It shows that

all activities are being performed to a larger extent to fulfill strategic intention A (Integrated

IT facilities for all students). Strategic intention B (Embedding IT in the education program)

is mainly supported by the entries of Applications and Infrastructure, while the entries of

Management as defined in that asset pool are also connected, but to a lesser degree. The same

applies to the entries of Service, although some services like enrollment of first-year students

and help desk / on-site support are more important than others to fulfill the strategic intention,

and therefore are better aligned to it.

Considering the third strategic intention C (Competitive advantage) just a few applications, all

entries in the infrastructure and most of the services are important and aligned. No entries

from the management asset pool are considered to be aligned with this strategic intention.

This is because the interviewees didn’t think these management processes are important to

achieve the desired strategic intention. In general the activities that are being performed to

achieve the strategic intentions are well aligned.

Next to the strategic alignment of the activities to the strategic intentions, the Internal

Alignment has to be performed. This practice contains the evaluation of the alignment

between applications and infrastructure; is the latter supportive for the used and needed

applications? According to the interviewees there is no misunderstanding about the alignment

of the infrastructure to the applications. Especially the Network LAN & Wireless, but also

internet and security are highly aligned. There are some exceptions: the Notebook

Information System is not a web application, so internet is not applicable and therefore not

aligned to it. Another exception is software licenses as an infrastructural entry; this is not

important for the Student Information System, so those are not aligned either.

49/85


3.7.2 Performance measurement

Like in Strategic alignment the Performance Measurement Practice consists of several parts.

The first part is the Cost Performance, and although cost reduction was not one of the

strategic intentions, one of the requirements was that the chosen alternative was to be feasible.

While not really stated, it was clear that cost control was one of the items of feasibility.

To compare the alternatives in the fulfillment of strategic intention A (Integrated IT facilities

for all students) the costs of the parts that would be different are compared. It seems of no

added value to determine the costs that are the same in both alternatives; it would only make

the comparison more complex.

To calculate the difference between the two probable alternatives, the costs of both desktop

PC and notebook, as far as they differ, are determined. Although the costs are estimated, a

significant difference is displayed. Part of the difference is ascribable to the fact that the costs

for the hardware are divided into a part to be paid for by the student and a part accountable to

the university. Other reasons are the lower service costs, explainable by the knowledge of the

students regarding IT facilities included the notebook, and the way service is delivered: carry-

in instead of on-site. A third cause for the higher costs of the desktop alternative is the

housing costs; in the case of a desktop workplace simply more space is needed. It is

calculated that the housing costs of the notebook alternative are 75% of that of the desktop

alternative.

If the desktop alternative was implemented the costs would be:

( numberofstudents − alreadyavailabledesktops)

* prizeofdesktop

= total costs

( 7000 − 500)

* 1.

06 =

6,

890,

000€

The costs of the notebook program, based on the described costs are:

numberofstudents * prizeofnotebook

= total costs

7000 * 745 = 5,

215,

000€

This means that, with a constant population of 7.000 students, the benefits of fulfilling

strategic intention A by means of the notebook program is 1,675,000€.

The second part of the Performance Measurement Practice consists of determining the Quality

and Service level of the asset pools of Service, Applications, and Infrastructure that were

specified. The objective of this practice is to provide management with the information they

might need to assign a larger or smaller budget to the various entries of the asset pools.

50/85


First, the Service level was examined. Even though there were some entries in the Service

asset pool that were not assigned the maximum score, there is no reason why extra resources

should be assigned to any of the entries.

The fact that the above mentioned services did not receive the highest score is explainable; the

services availability was not as high as that of other services. While determining the score,

together with the interviewees, it became clear that it was no problem that the enrollment of

first-year students did not take place 24 hours a day, every day. Other services that were

designated only occur over a short period in a year, so availability was not an issue, and

certainly not something a lot of extra resources had to be assigned to. As far as the services,

applications, and infrastructure that has to be available all the time there was no problem.

Measurements of availability and responsiveness all indicated that it was not necessary to

assign extra resources. But decreasing of the budgets was not advisable, because the

availability is crucial for the fulfilling of the strategic intentions.

Regarding the measurements of Quality the same analysis applies as presented for the Service

Level. All entries from the asset pools execute their tasks as they should. When less than the

maximum score was assigned, the interviewees stated that a deficiency was so small, that it

was not worth assigning extra resources to it. In two cases the interviewees knew that a large

amount of extra budget had to be assigned to add the extra functionality to the application.

They stated that the deficiency was not big enough to justify the extra resources.

Process Measurement is the third part of the Performance Measurement Practice, as already

stated in section 3.4.3 no process model was used while developing or monitoring the

notebook program. In spite of the lack of a process model, the processes that form part of the

notebook program were evolved to a fine-tuned set of processes that is transparent. Every

year these processes are evaluated, but little to no need for improvement has been found.

3.7.3 Communication

In this section the analysis of the survey among students of Eindhoven University of

Technology is presented. 98% of the respondents own a notebook. This means that students

that did not make use of the notebook program obtained a notebook elsewhere.

Every student makes use of his notebook for his education, 63% stated he uses it for 10-30

hours a week, 75% bring his notebook to the university two or more days a week for

education. Although 22% of the respondents declared that the notebook facilities on the

campus are insufficient, 65% stated differently; this group found the provided facilities

satisfactory.

51/85


As far as communication concerns similar questions were asked regarding the communication

among students, and between student and teacher. In the communication among students

personal contact is daily or weekly with 96% and thus the most used communication channel,

followed by email and telephone, that is used daily or weekly by 92% and 65% respectively.

Compared to the survey of 2000, the communication by email has increased; the

communication in personal contact or by telephone was not investigated in that survey.

The communication with teachers also increased, whereas in 2000 17% of the students stated

that they communicated by mail with their teacher, in the survey now 47% said that they

email on a daily or weekly basis. In the graph below a comparison is shown between the

surveys of 200 and 2008, concerning the communication by email. Note: in 2008 there was no

question regarding the communication with friends or by other means like the electronic

learning environment.

100

80

60

40

20

0

92

Percentage communication by email

70 68

47

17 16

Colleagues Friends Teachers Unknown

Fig. 3.9 Graph Percentage communication by email

4

Survey 2000

Survey 2008

Another outcome of the survey is the use of other electronic means of communication among

students or between student and teacher. Instant messenger is a popular way to communicate

among students, 80% said to use daily or weekly with colleague students. Instant messenger is

not used for communication between student and teacher. And although an Electronic

Learning Environment has been in use for quite some years, it is only used for communication

with the teacher by 30% of the respondents. Personal contact is the most common mean for

the communication between student and teacher, 52% of the respondents stated that this is

their preferred choice.

52/85


All these figures mean that the need for a good infrastructure for email and ELE exists, but

that personal contact between both students, and student and teacher is the most used and

preferred.

The same questionnaire contained four propositions that the respondents were asked to react

to. To the proposition “The notebook boosts the communication with your teacher more than

a desktop PC would do” 52% answered Agree or Agree Partially. Proposition two stated “The

notebook is going to play a larger role between student and teacher”, 66% answered Agree or

Agree Partially. To the proposition “The notebook is going to play a larger role between

students” the responses were 73% Agree or Agree Partially. The last proposition “I am

positive about the new communication means the university is going to offer me” was Agreed

or Partially Agreed to by 74% of the respondents. This implies that students expect an

increase in the use of notebooks in the educational programs and in communication in

general. An overview of the results of the survey is presented in appendix C.

3.7.4 Competitive advantage

The figures presented in section 3.6 show an increase of 346 first-year students during the

first two years after the initial implementation of the notebook program. The number of first-

year students has not decreased since then, which means that at that time a competitive

advantage occurred. Although the increase did not continue the benefits can be determined.

To make a proper calculation the costs also have to be included.

For both benefits and costs the number of 346 additional students is not completely correct,

30% of all students tend to stop with their studies. The costs and benefits therefore have to be

calculated with 243 students.

The costs for 243 additional students that make use of the notebook program are

243 * 745 = 181,

035€

. The benefits are, considering that the students

graduate, 243 * 31.

770 = 7,

720,

110€

, being the contribution from the authorities. This results

in a positive balance of 7,539,075€ in the first two years after the implementation of the

notebook program.

53/85


4 BENCHMARK BETWEEN FOUR UNIVERSITIES

4.1 Introduction

To conduct proper research on the effects of a notebook program like the existing one at

Eindhoven University of Technology, it is necessary to perform a benchmark on IT facilities

between several comparable organizations. For this benchmark a selection of universities took

place. The universities all have a special reason why they were chosen: Delft University of

Technology is a technical university, like Eindhoven University of Technology. The

education programs, besides their own specialties, are similar. The University of Groningen

was chosen because it is a large, general (that is not technical) university. Utrecht University

is also a large, general university; it was chosen because it is the only university that has

outsourced the operational management of IT facilities to an external organization.

Through the benchmark the differences between the various ways of providing students with

IT facilities will become apparent. After the presentation in section 4.3 of the data that are

collected through interviews with the directors of the IT organizations of all universities and

by examining the university websites, the comparison on personnel and financial aspects can

take place. The results of the research on those aspects are presented in sections 4.4 and 4.5.

4.2 Objective of the benchmark

The objective of the benchmark is to provide information with which a statement can be made

about the results of the notebook program of Eindhoven University of Technology.

This benchmark is not the main purpose of the research, which is why it is performed with

global facts and figures, especially in the parts that present the figures on personnel and

finances. To perform a benchmark that really has the purpose to rank the various

organizations, or to advise one or more organizations about improvements they have to make,

a more thorough investigation of the data has to be performed.

54/85


4.3 Universities included in the benchmark

In the following section a short description of the universities that are participating in the

benchmark will be given. Next to that is described at what way the various universities

provide their students with the IT facilities they need for their education program.

Eindhoven University of Technology

General information

The TU/e consists of 9 departments; together they provide 11 Bachelor and 26 Master’s

education programs. Furthermore, there are 4 Master’s programs to obtain a first-degree

teacher qualification, and 10 postgraduate designer programs, which are being provided in

cooperation with the technical universities of Delft and Twente.

In total there are at the moment about 7000 students, 240 professors, and several guest and

part-time teachers.

In addition the TU/e employs about 640 PhD students. Next to the scientific personnel there

are about 1170 support staff members. In total the TU/e employs about 3.000 scientific and

support staff members.

IT facilities

Eindhoven University of Technology does not have many computer rooms, in both public and

departmental buildings there are not more than 500 PC’s available for students. The TU/e

provides the students with a notebook program, in which the first-year students are equipped

with a notebook, but also with all the accessories, applications, and support they need to use

IT in their education program. They also have the disposal of IT facilities like for instance e-

mail, and an electronic learning environment. This means that, with the extension to the

notebook itself, at the TU/e the IT facilities are totally integrated and are so from the start of

the education. The university contributes in the costs of the notebook program. This

contribution of the notebook and the other facilities is in the form of an obligatory contract,

which the student can reduce by studying for three years at the university, after three years the

contract is redeemed. The other part is to be paid by the student himself. For students that do

not make use of the notebook program all the facilities, with exception of the service and

support of their notebook, are available as if they did make use of the program. For instance

all the software that is available for students with a “TU/e” notebook is also available for

students without a notebook, or a notebook they obtained elswhere.

55/85


With the notebook program for students and the availability of a computer for every member

of the scientific and support staff, the coverage of computers per person is 100% for both

students and employees.

University of Groningen

General information

The RUG consists of 9 departments; together they provide 61 Bachelor and 118 Master’s

education programs. Furthermore, there are 17 Research Masters, and 16 graduate schools.

In total there are at the moment about 24.000 students, 244 fte professors, and several guest

and part-time teachers. In addition the RUG employs 973 PhD students. In total the scientific

and support staff counts about 4100 staff members.

IT Facilities

The University of Groningen provides the students with computer rooms. In those rooms

there are more than 10.500 PC’s available for students, divided over public places like the

libraries, and departments. This means that 44% of the students in Groningen can work with a

computer simultaneously. All students get an e-mail address and other facilities like an

electronic learning environment at their disposal when they start their education. Support staff

members all have their own computer, so they have 100% of coverage.

The University of Groningen does not have a notebook program. If students would like to

have a notebook, they are being redirected to SURF (the collaboration of all the Dutch

universities). SURF offers all students and employees the possibility to obtain a notebook

against reduced costs. At the RUG there is no support available, except for problems with the

IT services that are provided by the IT service organization, like for instance email. These

services are not only provided to the students that have a notebook, but to all students and

employees. The services do not include solving problems with the notebook itself. If students

do have a notebook of their own, then they can use it through the wireless network that is

present in all the buildings of the university. The University of Groningen has no financial

involvement with the purchase of notebook by students.

56/85


Utrecht University

General

The UU consists of 7 faculties (including the University Medical Center) which contain 44

departments; together they provide 49 Bachelor and 196 Master’s education programs.

Furthermore, there are 6 graduate schools. In total there are at the moment about 29.000

students, 640 fte professors, several guest and part-time teachers, and about 440 PhD students.

Together scientific and support staff consist of about 8.600 staff members.

IT Facilities

The Utrecht University provides the students with computer rooms. In those rooms there are

about 2.750 PC’s available for students, divided over public places like the libraries, and

departments. This means that 9,5% of the students in Utrecht can work with a computer

simultaneously. All students get an e-mail address and other facilities like an electronic

learning environment at their disposal when the start their education.

Scientific and support staff members all have their own computer, so they have 100% of

cover

The Utrecht University does not have a university wide notebook program. One of the

faculties does offer services for notebooks that are purchased through a special contract the

faculty has with a manufacturer.

If students have their own notebook, then they can use it through the wireless network that is

present in most of the buildings of the university. The Utrecht University has no financial

involvement with the purchase of notebook by students.

Delft University of Technology

General

The Delft University of Technology consists of 8 departments; together they provide 14

Bachelor and 44 Master’s education programs of which 10 are provided through collaboration

with the universities of Rotterdam, Eindhoven, Twente, and Leiden. In total there are at the

moment about 15.000 students, 197 fte professors and about 2.500 persons other scientific

staffs. Next to the scientific staff there are about 1.850 support staff members.

Although most departments participate in a notebook project, it is not an university wide

program.

57/85


IT Facilities

Delft University of Technology provides about 3000 PC’s in computer rooms, scattered over

public and departmental buildings. Next to that about 1.500 students until this moment have

participated in the notebook project that is offered by the university. This means that about

4.500 PC’s are available for 15.000 students, which means 32% coverage of IT means.

Scientific and support staff members all have their own computer, so they have 100% of

coverage. The notebook project of the TU Delft consists of the choice of the notebook in

collaboration between the IT organization and the participating departments, the build of an

image based on the university workplace, and a helpdesk at which students can come with

their notebook problems (hard- and software). Like in Eindhoven, in case of a broken

notebook the student gets another (temporary) notebook. Software support consists of the

availability of a departmental image, so when a student has a software problem he can bring

his software back in the original state. Like on the other universities all students get IT

facilities like an e-mail address and access to the electronic learning environment.

In the table below the data that are described above are summarized.

Students

Staff

Available PC’s for

students

Available PC’s for

employees

Total number

of PC’s serviced

Percentage of

coverage students

Percentage of

coverage

employees

Notebook program

Eindhoven University

of Technology

University of

Groningen

Utrecht University Delft University of

Technology

7.000 24.000 29.000 15.000

3.000 4.100 8.600 4.440

7.000 10.500 2.750 4.500

3.000 4.100 8.600 4.440

10.000 14.600 11.350 8.950

100 % 44 % 9,5 % 32 %

100 % 100 % 100 % 100 %

Fully integrated Not available

Table 4.1 summarized data four universities

One faculty, no

subsidy

Partly, only subsidy

on RSI and helpdesk

58/85


4.4 Personnel benchmark

One of the factors that make up the total costs of facilitating students and employees with IT

facilities is personnel. Without the people who decide what is to be delivered, and without the

personnel actually delivering the services there is no availability of IT facilities. Because the

costs for personnel are by far the most costly component of service these figures are presented

separately. The personnel costs are in all universities the better half of the total costs of the IT

service organization. In the table below the number of personnel that is employed by the IT

organization is added.

Students

Staff

Staff IT

Organization

Available PC’s for

students

Available PC’s for

employees

Total number

of PC’s serviced

Percentage of

coverage students

Percentage of

coverage

employees

Notebook program

Eindhoven

University of

Technology

University of

Groningen

Utrecht University Delft University of

Technology

7.000 24.000 29.000 15.000

3.000 4.100 8.600 4.440

81 142

95 (excl.

outsourcing)

7.000 10.500 2.750 4.500

3.000 4.100 8.600 4.440

10.000 14.600 11.350 8.950

100 % 44 % 9,5 % 32 %

100 % 100 % 100 % 100 %

Fully integrated Not available

One faculty, no

financial

involvement

Table 4.2 Summarized data four universities, incl. IT personnel

205

Partly, only subsidy

on RSI and

helpdesk

In the above table the only university that is incomparable, regarding to number of full time

equivalent staff is Utrecht University.

59/85


Indicators of the comparison regarding the personnel aspect of the IT organization are the

number of people that is responsible for the IT facilities, and the number of supported PC’s.

Of course the number of people needed is dependent of the number of PC’s, but there is also a

dependency of the services that the organization delivers to its customers. Although there is

some difference between the services that are delivered, most of the services are equal. For

instance: Groningen also provides in IT applications courses, but doesn’t have a notebook

service desk. On the other hand: Eindhoven has an extended service for students with a

notebook, and does not provide in applications courses. One figure is very divergent: the one

of Delft University of Technology. A probable reason is the extended role that the IT

organization in Delft has regarding the development and implementation of information

systems for research and educational purposes. Next to that the IT organization in Delft also

develops information systems for the university library. If all the services the various

universities, with the exception of Delft, deliver are being weighed against each other, the

offer of services is equal. This means that the above mentioned indicators can give a

reasonable idea of the performance of the various IT organizations. The ratios are summarized

in the table below.

Total number

of PC’s serviced

Staff

IT Organization in

fte

Ratio PC / fte

Eindhoven

University of

Technology

University of

Groningen

Utrecht University Delft University of

Technology

10.000 14.600 11.350 8.950

81 142

95 (excl.

outsourcing)

123 102 120 43

Table 4.3 Ratios PC’s per fte IT organization

The other universities do not have that extended role; they are restricted to manage the

availability of the technical infrastructure (including the PC’s and other workplaces), and the

development on the organizational systems like the corporate financial or salary system.

205

60/85


The development of information systems for research, education and library are not part of

their assignment. This means that the universities besides Delft are comparable regarding the

activities the IT service organizations perform with the personnel they employ.

To make the figures between the universities comparable, the number of employees of

Utrecht is adapted. Because Groningen is a comparable, general, university those figures was

the basis for the adaptation. The ratio PC / fte of Groningen is taken to calculate the needed

number of support staff for Utrecht.

The adaptation results in the table below.

Total number

of PC’s serviced

Staff

IT Organization in

fte

Ratio PC / fte

Eindhoven

University of

Technology

4.5 Financial benchmark

University of

Groningen

Utrecht University Delft University of

Technology

10.000 14.600 11.350 8.950

81 142 110 205

123 102 103 43

Table 4.4 Ratios PC’s per fte IT organization, adaptation to Utrecht

Although the financial data of all four universities is available, it is even more difficult to

compare than the data in the previous section. Again it is clear that the way the budgets are

being spent is not very similar. For instance: in Delft the budget includes the revenues of an

IT shop, whilst in Eindhoven the costs of telephony facilities have to be taken into account. In

Utrecht the budget of all IT facilities is embedded with the central IT organization, whereas in

Groningen the departments and other organizational units have their own IT budgets.

Therefore it seems hardly possible to compare the figures in a proper way. To have an idea of

what these figures are they are summarized in the table below.

61/85


University budget

(million € )

IT budget

(million € )

Ratio budgets

Eindhoven University

of Technology

University of

Groningen

Utrecht University Delft University of

Technology

297 520 695 487

10 12 26 37

3.36% 2.30% 3.74% 7.60%

Table 4.5 Budgets of the universities

If looked at the figures, the difference seen in the previous section is also obvious here. Delft

University of Technology has more budgets available. It seems corresponding with the

conclusion that Delft has more personnel because they have more activities they perform next

to keeping the technical infrastructure available.

4.6 Analysis

The benchmark was performed as part of the evaluation of the notebook program of

Eindhoven University of Technology. The objective was to determine whether the program

caused an alteration of personnel or financial resources.

Considering the personnel aspects it is clear that the activities the IT service organization

performs are of great influence on the number of personnel that is needed. In the interviews

that were held with the directors of the various universities it became obvious that most of the

activities that are performed are identical, only Delft University of Technology has a deviant

portfolio of services. Of course the number of work and study places that is to be supported,

also affects the needed personnel. Another probable reason for the difference in number of

personnel is the way the IT infrastructure and facilities are implemented. In Eindhoven every

employee is the administrator of his own computer system. This means that he for instance

has the opportunity to install applications, or remove them as he sees fit. In Groningen the IT

service organization composed a university workplace, at which the user of the system can

not change anything. The latter method results in a lower support load for the IT service

organization.

62/85


Concerning the finances, the following statements can be made, bearing in mind that it is

difficult to compare because the activities and especially the way the budget is calculated can

be very different. The comparison is based on the percentages of the total budget, not on the

absolute figures. Because salaries in all IT service organizations account for well over 50% of

the budget, the number of employees is of great influence on the total budget. In the case of

the benchmark between the four universities a difference is seen between Delft and the other

universities, probably ascribable to the extended portfolio of services. This difference, in

accordance with the above remarks, is visible in the personnel numbers. Some reservation has

to be made in the comparison between the other three universities because the exact number

of personnel for Utrecht University is not known.

The budget of Delft University of Technology is substantial compared to the other

universities. This is probably caused by the number of personnel Delft has to employ to

deliver the services as described in the previous sections. Next to that, Delft is a technical

university. Research and education programs at technical universities in most cases demand

higher specifications of the systems that are being used, as well as more specialized support

staff.

The latter reasons are probably also the reasons for the relatively high budget of Eindhoven in

comparison to that of Groningen.

The University of Groningen has the lowest budget; it could be possible that this is due to the

characteristics of a general university. Most research and education programs on a general

university do not demand highly specified equipment and corresponding support staff. And

although that in some fields of study this is not entirely true, it can be an indication for the

lower budget. Utrecht, on the other hand has a relatively high budget in comparison to

Groningen, perhaps due to the mixed character of the educational program. Utrecht is a

general university, except for the Beta faculty, which consists of six departments which have

the same characteristics as the departments of a technical university. Besides that, part of the

service organization is outsourced; perhaps those costs also increase the budget.

63/85


5 CONCLUSIONS AND RECOMMENDATIONS

5.1 Introduction

In this section the results of the evaluation and the benchmark are presented. Next to that the

research question and sub-research questions are answered. The final section of this chapter

contains recommendations regarding adaptation of the used evaluation method.

5.2 Results of the evaluation

The objective of the evaluation of the notebook program at Eindhoven University of

Technology was twofold. The main goal was to serve as a case study to answer the first part

of the research question: With which possible methods can the notebook program of the

University of Technology of Eindhoven be evaluated?

But the second part of the research question: What are the results of that evaluation? also was

important, especially for the supervisor of Eindhoven University of Technology.

In this section the results of the evaluation itself are presented, the remarks and

recommendations regarding the used method are presented in section 5.5.

The main result of the evaluation regarding the case study on the notebook program is the

conclusion that the university’s strategic statement which was made in 1997 is fulfilled. The

assignment was to achieve integral IT facilities for both students and employees of the

university. This meant that they could have the IT facilities that they felt where necessary at

their disposal whenever they needed them. This strategic statement was realized with the use

of notebooks, the technical infrastructure, and the organizational processes to support the

availability of integral IT facilities. Because all students and employees (100%) have the IT

facilities available at all times these facilities can be appointed integrally. Next to the

conclusion that the notebook program fulfilled the strategic statement, the program itself was

the subject of further evaluation. The strategic alignment with the strategic intentions that

were extracted form the above mentioned statement is proven, and with further investigations

based on the chosen method an evaluation was performed on the effects of the notebook

program. The strategic intentions were: A. Integrated IT facilities for all students and

employees in a feasible way, B. Embedding IT in the education program, and C. Acquire

competitive advantage compared to other universities in the Netherlands. The effects were

successively the feasibility of the program, the effects on both the communication between

students and their teachers and students amongst each other, and possible competitive

advantage in comparison with other (technical) universities.

64/85


The results as found for the first effect were surprising, with the provisioning of students with

notebooks against subsidiary costs, the university created a cost reduction instead of the

expected increase. If the alternative of placing desktop computers everywhere on the campus

was implemented the costs would be higher.

A few remarks are needed: the initial costs for the development of the program are not

included, and the costs for both desktop and notebook version are based on the prices and

rates at the moment. The second strategic intention that was evaluated was the effect the

notebook program had on the way the communication was done. This was no surprise; in the

early years of the notebook program communication by email was already high, especially

among students. In the survey that was performed for this evaluation, the use of email only

increased, not only among students but also between students and teachers. Next to that, the

survey indicated an increasing use of other electronic communication means like Instant

Messenger. It was obvious that the use of notebooks, and all the applications that were

developed to support the notebook program increased the use of IT facilities in the education

programs. And last but not least, the third strategic intention: competitive advantage. The

evaluation at the numbers of Eindhoven University of Technology was clear: during the first

two years of the notebook program the number of first-year student increased by 346 students

and that number did not decrease in the years following. To complete the evaluation of the

competitive advantage it was necessary to compare these figures with the figures from the

other technical universities. The benchmark that was performed showed that, in spite of the

existence of a kind of notebook project at the Delft University of Technology and Twente

University, they did not have a competitive advantage ascribable to that program. The

evaluation at TU/e, not only concerning the number of first-year students, but also regarding

the other effects shows that a notebook program is more than the providing of notebooks to

students. Besides the notebook, all the supportive processes, the underlying infrastructure, but

most of all the use of these facilities in the education programs is what has made it a success.

The results of the evaluation are shown in the figure below, in which the objectives are

presented, and the corresponding table with the recapitulated data. Because there are no

financial benefits assigned to the middle ring of the model, the communication between

student and teacher, that aspect is not mentioned in the table.

65/85


Competitive advantage TU/e

Changes in communication between

student and teacher

Financial aspects

for students & university

5.3 Result of the benchmark

Fig. 5.1 Model and results of the evaluation

Financial aspects 1,675,000€

Competitive advantage 7,539,075€

Total benefits TU/e 9,241,075€

The objective of the benchmark was to determine whether the notebook program of

Eindhoven University of Technology caused a difference in personnel or financial resources.

Possible causes of the difference in numbers of personnel are the type of university (general

versus technical), the portfolio of services they offer, and the method of implementation of the

IT facilities.

Although these reasons are not easy to clarify in numbers, the interviews that were conducted

in this benchmark and the research done on the websites of the participating universities gave

insight into the various ways IT service organizations supported their customers, as well in

the portfolio of services.

The result of the benchmark on personnel is the conclusion that there is a difference between

the numbers of IT personnel the participating universities employ. Looking at the figures of

Eindhoven the following conclusion can be drawn: although it is not obvious, considering all

the aspects that should increase the number of personnel, Eindhoven University of

Technology delivers the services to their customers with the least number of personnel. The

reasons for that outcome can be divers. One reason is that the students have the knowledge to

act as the administrators of their own notebook and have facilities at their disposal to do so.

66/85


Another reason is that if a problem cannot be solved by the student the service support

employee does not have to go to the workplace, because it is a notebook it can be transferred

to the support employee.

As far as the financial benchmark is concerned, it is clear that the salaries take up a large

portion of the IT budget. But also the portfolio of services, like with the personnel benchmark,

and the components that the budget consists of are of influence. The data for the financial

benchmark that were obtained during the interviews are very different, as stated before: for

instance, the inclusion of telephony in the budget can make a significant difference.

Therefore it is nearly impossible to extract a result from the financial benchmark, and

impossible to do so with just arguments.

5.4 Answering the research questions

5.4.1 Introduction

In this section the research question and sub questions as stated in section 1.5.2 are answered.

Some of the sub questions are already extensively answered for in the various parts of this

report, for those questions a recapitulation and a reference to the relevant chapters and

sections will be given.

To finally come to the answering of the research question, first all sub questions must be

answered, in order to come to a proper answer on the main research question.

5.4.2 Answering the sub questions

Sub question 1. What are the existing evaluation methods?

The existing methods can be divided in different categories; each with its own approach,

advantages, and disadvantages. A distinction can be made between:

• financial approach

• multi-criteria approach

• ratio approach

• portfolio approach

A description of these approaches, their objectives, and their strengths and weaknesses is

presented in chapter two. Also a choice for a method is made to perform the evaluation on the

notebook program of Eindhoven University of Technology, that specific method is described

in more detail in the same chapter.

67/85


Sub question 2. What comprises the notebook program at the TU/e?

The notebook program of Eindhoven University of Technology is the implementation of a

strategic choice of the university to provide all persons that are studying or working on the

university grounds, and if necessary outside the campus boundaries, with integral IT facilities.

This meant that every student and employee had to be provided with IT facilities wherever

and whenever they needed them. To reach that objective a project team was installed, who

came with a solution that made use of notebooks. In short: students have the opportunity to

make use of the notebook program when they (for the first time) enroll in a TU/e education

program. The notebook program consists of IT facilities like email, data storage, and an

electronic learning environment. Part of the program is of course the notebook itself, with a

complete set of accessories and services. This package is offered to the students at low costs,

with a contribution from the university, so that as many students as possible can make use of

the program. As a result of this way of offering IT facilities, 95% of the universities first-year

students make use of the program. An extended description of the notebook program is

presented in section 1.4

Sub question 3. What are the objectives that were intended with the notebook

program?

The objectives of the notebook program are described in section 1.4.2 and summarized in a

table in section 3.2 where the evaluation on the alignment between notebook program and the

objectives is being performed.

Extracted from the description and the table the objectives are:

• Integrated IT facilities for all students and employees in a feasible way;

• Embedding IT in the education program;

• Acquire competitive advantage compared to other universities in the Netherlands.

Sub question 4. Which method is the most suitable for the evaluation of the notebook

program?

The result of the literature research on the various IT evaluation methods was the choice of

New Information Economics (NIE) as the method to evaluate the notebook program of

Eindhoven University of Technology. NIE is a method in the category Multi-Criteria

approaches.

This method is chosen because it emphasizes the importance of alignment of IT with the

strategic intentions of the organization.

68/85


In an evaluation like the one on the notebook program, where strategic objectives are the main

reason to implement IT facilities, this alignment is the far most important issue to evaluate.

Next to the Alignment Practice, as it is being called in NIE, a Performance Measurement

Practice is also presented, which is divided in financial and other performance measurements.

NIE consists of more practices, like Innovation, Prioritization, and Strategic Demand/Supply

Planning Practices, which makes it suitable for ex ante evaluations. But the practices are well

suited to perform separately, which makes the method fit to use for an ex post evaluation.

Sub question 5. What is the result of the evaluation?

The data and analysis of the evaluation is presented in chapter three; the benchmark with three

other universities is presented in chapter four. Although a benchmark was not necessary to

perform the evaluation of the notebook program itself, it was interesting to conduct a research

on the same aspects at the other universities to make a statement about the performances of

Eindhoven University of Technology.

Sub question 6. How can the used method be evaluated?

The evaluation of the used method is performed by applying the method on the notebook

program of Eindhoven University of Technology. While applying the method possible

deficiencies will occur. Some can, and did, already occur during the deepening of the

literature regarding the literature. The conclusions of those findings are presented in the

answering of the next sub question, which is more extended in section 5.4 of the report.

Sub question 7. Are there recommendations to adapt the chosen method to make it

more suitable?

Because the New Information Economic method is developed as an ex ante IT investment

evaluation method, there are some elements that are not suitable for an ex post evaluation.

Specifically the Strategic demand/Supply planning, Prioritization, and Innovation Practices

are not applicable. The arguments that lie below this statement are presented in section 5.4.

It would be recommended to remove these practices from the method, in case of an ex post

evaluation.

The remaining practices, Alignment and Performance Measurement, are applicable. Like in

every method, it depends on the subject of the evaluation whether every element of these

practices can of have to be used. Adaptation of the method as described is not recommended,

the method is meant to be applicable for several kinds of evaluations.

69/85


On basis of the extended version it could be recommended to develop a less extended version,

especially for straight forward ex post evaluations. Just adapting the extended version would

mean that it is applicable for only a restricted number of evaluations. For recommendations

on adaptation especially in case of the evaluation of the notebook program refer to section 5.4

With the answering of all above sub research questions the main research question can be

answered as well.

5.4.3 Answering the main research question

What is the result of the evaluation of the notebook program at Eindhoven University

of Technology after the selection of a suitable method?

Possible evaluation methods are part of the category multi-criteria approaches. Most of these

methods are especially developed for the evaluation of IT investments, but can also be used

for ex post evaluations like the one on the notebook program.

The choice for New Information Economics is made because this method emphasizes the

importance of strategic alignment, and is also of use in performance measurements.

For the data and the analysis on the evaluation refer to the extended presentation in chapter

three and the results in sections 5.2.

5.5 Recommendations regarding the used method

As stated in the answers of the previous sub-question 6, there were two occasions in which

possible gaps or unnecessary elements could be distinguished. First, while studying the

literature regarding the method, and second while performing the evaluation itself.

Probably because the method has developed as an ex ante method for IT evaluation, during

the studying of the literature it became apparent that three of the five practices of the method

were not applicable. The first one: Strategic demand/Supply planning can not be applied in an

ex post evaluation. This practice builds a consensus view on how IT is to be used. In the case

of an ex post evaluation, this consensus is already reached, preferably in an ex ante

evaluation. An ex post evaluation is about measuring the results of the use of IT, not the

intentions of how to use it. The second practice that is not suitable to use in an ex post

evaluation is Prioritization.

70/85


The main objective of this practice is to determine which IS/IT system out of several that are

being evaluated is to be implemented. It is obvious that this practice is not applicable in an ex

post evaluation; the choice for a system was already made in the past. The third practice that

is hardly useful in an ex post evaluation is Innovation. This practice translate new IT

opportunities into competitive advantage and bottom-line results. This means that IT is

leading, instead of supportive, and that a choice has to be made with system that is the most

innovative. This is again an ex ante investment decision.

The evaluation on the notebook program of Eindhoven University of Technology was

therefore performed with the practices Strategic Alignment, and Performance Measurements.

Both practices have a part called “Functional alignment”, which is much alike; the functional

alignment of the Strategic Alignment Practice measures the service levels and qualities of

applications, services, and infrastructure, so does the Performance Measurement Practice.

Although there are some differences, for the evaluation of the notebook program the former

did not seem appropriate because the alignment of the program had already been measured.

Due to the overlap in the measurements in both the Strategic Alignment and Performance

Measurement Practices, it seemed that a choice for either one had to be made.

It seems that the three mentioned practices in the first part of this section are not suitable for

an ex post evaluation, so it is recommended to leave them out when using the New

Information Economics method for an ex post IT evaluation.

As far as the remaining practices concerns it would be recommendable to merge the

Functional Alignment parts of both practices. Although they both have their own basic

assumptions, a merger seems possible. In that way a separate method for ex post evaluations

can be developed.

71/85


6 DISCUSSION AND REFLECTION

Now that the evaluation and benchmark have been presented, the conclusions have been

drawn, and the recommendations were stated, some remarks about the research are in place.

In the first place remarks are given concerning the research itself, besides that reflection on

the process I have gone through conducting the research and writing this report. The latter to

learn about the main difficulties I encountered during the research.

The evaluation that was performed on the notebook program has a positive outcome

concerning the realization of the strategic statements of 1997, but a restriction in this matter is

in place. Due to the time factor more and more students own a notebook, not necessarily

obtained make making use o a notebook program or notebook project. When walking through

the buildings of the various universities I visited for interviews, I witnessed a lot of students

working with notebooks. If universities should decide to utilize those notebooks in the

educational programs, they don’t need to invest in subsidies for hardware, but can restrict the

costs to the ones that are needed to implement the necessary infrastructural and educational

adaptations.

Another reservation has to be made concerning the alternative of placing desktop PC’s to

implement the integral IT facilities. It perhaps is not necessary to place as many desktops as

there are students. Most of the time not all students are present at the university. But then

again; if the strategic statement was to provide IT facilities any time and any place, facilities

for studying at another place (i.e. at home) should be provided.

Reservations also have to be made regarding the conclusions of the benchmark. Like already

stated in section 4.6 the data which form the foundation of the analyses of the benchmark and

the subsequent conclusions are rough figures. In spite of that observation, I think the results

fairly reflect reality. Another remark has to be made regarding the number of first-year

students. Although the number in Eindhoven increased during the first years of the notebook

program, they did not decrease at the other technical universities. It has been an assumption to

ascribe the increase to the notebook program, but hard evidence is hard to produce to endorse

this assumption.

The last comment on this research is that due to the limited time, the cultural aspects of the

evaluation are not included. Evaluating that aspect probably would have given insight into

how organizations cope with a change caused by the implementation of such a far-reaching

strategic intention.

72/85


The process of conducting this research and writing the thesis was new for me, at least in this

way. Notwithstanding all the support I received from my supervisors and all the other people

that I could depend on, most of the brainwork was up to me. I realize that this is part of the

academic education, but it caused some trouble along the way. First of all the planning; no

colleges to prepare, what makes it easy to go on because an unprepared college is a waste of

time. But to make your own planning and to live up to it is less easy, the latter being the hard

part. Work and other nice things to do always seem to be more important.

I learned, with help and encouragement of a lot of people, that structure is important. It helps

you to go on, although sometimes it is not so trivial.

The part of the research that was the most difficult was the quest for literature. The world of

articles and books is immense, and there are so many interesting subjects. Especially the latter

made it difficult to keep focused on the right literature. The evaluation and the benchmark

were the (relatively) easy parts; it was interesting to see what the effects of the notebook

program are, and if Eindhoven University of Technology had done things worse or better.

Nevertheless I enjoyed conducting the research, and I know it added value to my academic

education, as well as to my personal awareness.

73/85


APPENDIX A REFERENCES

Al-Yaseen, H., Eldabi, T., Lees, D. Y., & Paul, R. J. 2006. Operational Use evaluation of IT

investments: An investigation into potential benefits. European Journal of Operational

Research, 173(3): 1000.

Baarda, D.B. & de Goede, M.P.M. 2001 Basisboek Methoden en Technieken. Stenfert Kroese,

Groningen

Bacon, C. J. 1992. The Use of Decision Criteria in Selecting Information

Systems/Technology Investments. MIS Quarterly, 16(3): 335.

Bakos, J. Y. & Treacy, M. E. 1986. Information Technology and Corporate Strategy: A

Research Perspective. MIS Quarterly, 10(2): 107.

Ballantine, J., Levy, M., Martin, A., Munro, I., & Powell, P. 2000. An ethical perspective on

information systems evaluation. International Journal of Agile Management Systems, 2(3):

233

Barnes, D. 2001. The hidden costs of implementing and maintaining information systems. The

Bottom Line, 14(4): 207.

Barreau, D. 2001. The hidden costs of implementing and maintaining information systems.

The Bottom Line, 14(4): 207.

Bennington, P. & Baccarini, D. 2004. Project benefits Management in IT Projects-an

Australian Perspective. Project Management Journal, 35(2): 20.

Benson, R.J., Bugnitz, T.L. & Walton, W.B. 2004. From Business Strategy to IT Action.

Hoboken USA: John Wiley & Sons

Berghout, E. W. 1997. Evaluation of Information System Proposals: Design of a Decision

Support Method. Technische Universiteit Delft.

Berghout, E. W. & Renkema, T. W. J. 2005. Investment Assesment of IT-projects. Rotterdam:

Information Management Institute.

74/85


Berghout, E. W. and Remenyi, D. The Eleven Years of the European Conference on IT

Evaluation: Tertrospectives and Perspectives for Possible Future Research. The Electronic

Journal of Information Systems Evaluation 8[2], 81-98. 2005.

Bharadwaj, A. S. 2000. A resource-based perspective on information technology capability

and firm performance: An empirical investigation. MIS Quarterly, 24(1): 169.

Braaksma, J., Commandeur, A., and Berghout, E. W. The Business Case for (ICT) Investment

Evaluation in Nonprofit Organizations. 2008. Groningen, Centre of IT Economic research.

Chen, Y., Liang, L., Yang, F., & Zhu, J. 2006. Evaluation of information technology

investment: a data envelopment analysis approach. Computers & Operations Research, 33(5):

1368.

Dewan, S. & Ren, F. 2007. Risk and Return of Information Technology Initiatives: Evidence

from Electronic Commerce Announcements. Information Systems Research, 18(4): 370-394.

Fisher, A. 1995. Information economics: A new paradigm for information management and

technology decision-making. The Government Accountants Journal, 44(2): 30.

Golden, W. & Powell, Ph. 2004. Inter-organisational Information Systems as Enablers of

Organisational Flexibility. Technology Analysis & Strategic Management, 16(3): 299.

Gordillo, G. & Andersson, K. 2004. From policy lessons to policy actions: motivation to take

evaluation seriously. Public Administration & Development, 24(4): 305.

Grembergen, W. v. 2001. Information Technology Evaluation Methods & Management.

Hershey: Idea Group Publishing.

Hajer, K. 2007. Using a Systems Thinking Perspective to Construct and Apply an Evaluation

Approach of Technology-Based Information Systems. Information Resources Management

Journal, 20(2): 108.

Hamilton, S. & Chervany, N. L. 1981. Evaluating Information System Effectiveness - Part I:

Comparing Evaluation Approaches. MIS Quarterly, 5(3): 55-69.

75/85


Hitt, L. M. & Brynjolfsson, E. 1996. Productivity, business profitability, and consumer

surplus: Three different measures of information technology value. MIS Quarterly, 20(2):

121.

In, L. 2004. Evaluating business process-integrated information technology investment.

Business Process Management Journal, 10(2): 214.

Irani, Z. & Love, P. E. D. 2002. Developing a frame of reference for ex ante IT/IS investment

evaluation. European Journal of Information Systems, 11(1): 74.

Irani, Z., Sharif, A. M., & Love, P. E. D. 2005. Linking knowledge transformation to

Information Systems evaluation. European Journal of Information Systems, 14(3): 213.

Irani, Z., Ghoneim, A., & Love, P. E. D. 2006. Evaluating cost taxonomies for information

systems management. European Journal of Operational Research, 173(3): 1103.

Irani, Z., Sharif, A. M., & Love, E. D. 2007. Knowledge mapping for information systems

evaluation in manufacturing. International Journal of Production Research, 45(11): 2435.

Jiang, J. J., Klein, G., & Carr, C. L. 2002. Measuring information system service quality:

SERVQUAL from the other side. MIS Quarterly, 26(2): 145.

Jonker, J. & Pennink, B.W.J. 2004. De kern van methodologie. Van Gorcum, Assen

Kohli, R. & Hoadley, E. 2006. Towards Developing a Framework for Measuring

Organizational Impact of IT- Enabled BPR: Case Studies of Three Firms. Database for

Advances in Information Systems, 37(1): 40.

Kumar Sharma, M. & Bhagwat, R. 2006. Performance measurements in the implementation

of information systems in small and medium-sized enterprises: a framework and empirical

analysis. Measuring Business Excellence, 10(4): 8.

Kusters, R. J. and Renkema, T. W. J. Designing Local Evaluation Models for IT Investments.

1994. Eindhoven, Eindhoven University of Technology.

Leeuw de, A.C.J., 2003 Management van onderzoek. Van Gorcum, Assen

76/85


Lin, C., Huang, Y., Cheng, M., & Lin, W. 2007. Effects of Information Technology Maturity

on the Adoption of Investment Evaluation Methodologies: A Survey of Large Australian

Organizations. International Journal of Management, 24(4): 697.

Love, P. E. D. & Irani, Z. 2004. An exploratory study of information technology evaluation

and benefits management practices of SMEs in the construction industry. Information &

Management, 42(1): 227.

Love, P. E. D., Ghoneim, A., & Irani, Z. 2004. Information technology evaluation: classifying

indirect costs using the structured case method. Journal of Enterprise Information

Management, 17(4): 312.

Love, P. E. D., Irani, Z., Standing, G., Lin, C., & Burn, J. M. 2005. The enigma of evaluation:

benefits, costs and risks of IT in Australian small-medium-sized enterprises. Information &

Management, 42(7): 947.

Love, P. E. D., Irani, Z., Ghoneim, A., & Themistocleous, M. 2006. An exploratory study of

indirect ICT costs using the structured case method. International Journal of Information

Management, 26(2): 167.

Luca, H. C. 2005. Information Technology, Strategic Decision Making for Managers.

Hoboken USA: John Wiley & Sons, Inc.

Luftman, J. & Muller, H. 2005. Total Value Of Ownership: A New Model. Optimize, 4(7):

51.

Mata, F. J., Fuerst, W. L., & Barney, J. B. 1995. Information technology and sustained

competitive advantage: A resource-based analysis. MIS Quarterly, 19(4): 487.

Parker, M. M., Benson, J. R., & Trainor, H. E. 1988. Information Economics. New Yersey:

Prentice-Hall International Inc.

Parker, M. M., Trainor, H. E., & Benson, J. R. 1989. Information Strategy and Economics.

London: Prentice-Hall International Inc.

Patel, N. V. and Irani, Z. Evaluating Information Technology in Dynamic Environments: a

Focus on Tailorable Information Systems. Logistics Information Management 12[1/2], 32-39.

1999.

77/85


Powell, Ph. 1992. Information Technology Evaluation: Is It Different? The Journal of the

Operational Research Society, 43(1): 29-42.

Powell, T.C. & Dent-Micallef, A. 1997. Information as Competitive Advantage: the Role of

Human, Business, and Technology Resources. Strategic Management Journal (1986-1998),

18(5): 375.

Renkema, T. J. W. & Berghout, E. W. 1997. Methodologies for information systems

investment evaluation at the proposal stage: A comparative review. Information and Software

Technology, 39(1): 1.

Renkema, T. W. J. 1994. Besluitvorming over Investeringen in de Informatie-infrastructuur.

www.informatie.nl .

Renkema, T. W. J. and Berghout, E. W. Investeringsbeoordeling- en management. 2005.

Groningen, Centre of IT Economic Research.

Rodney, S. & Sherif, M. 2002. IT/IS projects selection using multi-criteria utility theory.

Logistics Information Management , 15(4): 254.

Rose, J. M., Rose, A. M., & Strand Norman, C. 2004. The Evaluation of Risky Information

Technology Investment Decisions. Journal of Information Systems, 18(1): 53.

Salmivalli, L., Hypponen, H., Nykanen, P., Ruotsalainen, P., & Pajukoski, M. 2007. Testing a

theoretical framework for interdisciplinary IT evaluation: The case of Finnish Electronic

Prescription. International Journal of Healthcare Technology & Management, 8(1/2): 42.

Serafeimidin, V and Smithson, S. Rethinking the Approaches to Information Systems

Investment Evaluation. Logistics Information Management 12[1/2], 94-107. 1999.

Standing, G., Guilfoyle, A., Lin, C., & Love, P. E. D. 2006. The attribution of success and

failure in IT projects. Industrial Management + Data Systems, 106(8): 1148.

Stewart, R. & Mohamed, S. 2002. IT/IS projects selection using multi-criteria utility theory.

Logistics Information Management, 15(4): 254.

Suwardy, T. &. R. J. & Sohal, S. S. 2003. IT projects: Evaluation, outcomes and

impediments. Benchmarking, 10(4): 325.

78/85


Tavakolian, H. 1989. Linking the Information Technology Structure with Organizations MIS

Quarterly, 13(3): 309.

Weill, P. & Vitale, M. 1999. Assessing the health of an information systems applications

portfolio: An example from process manufacturing. MIS Quarterly, 23(4): 601.

Willcocks, L. 1992. It evaluation: Managing the catch 22. European Management Journal,

10(2): 220-229.

Willcocks, L. & Lester, S. 1991 Information Systems Investments: Evaluation at the

Feasibility Stage of Projects. Technovation, 11(5): 283-302

Unknown authors

Research and Markets: Developing the IT Scorecard 2004. Business Wire, 1.

The Advancement of IS Evaluation, a Literature Review. 2006.

Websites

http://www.tue.nl http://www.rug.nl

http://www.uu.nl http://www.tudelft.nl

http://www.utwente.nl

Internal documents Eindhoven University of Technology

Documents of the initial project, reports of meetings etc.

Multimedia Resources for Chemistry in Europe. Amsterdam, 1999

Notebooks and Students, Experiences of three generations notebook students at Eindhoven

University of Technology. Eindhoven, 2000

Notebook Survey, Research to the use and experiences with notebooks. Eindhoven, 2004

79/85


APPENDIX B LIST OF FIGURES AND TABLES

Figure 1.1 Organization Chart of TU/e ................................................. 9

Figure 1.2 Timeline notebook program delivery 2008......................... 14

Figure 2.1 Strategy-to-Bottom-Line Value Chain ............................... 30

Figure 3.1 Strategic intentions notebook program............................... 33

Figure 3.2 Strategic Alignment........................................................... 35

Figure3.3 Internal Alignment............................................................. 37

Figure 3.4 Performance Dimensions................................................... 38

Figure 3.5 Graph of communication by notebook survey 2000 ........... 45

Figure 3.6 Graph student number Eindhoven...................................... 46

Figure 3.7 Graph student number Delft............................................... 47

Figure 3.8 Graph student number Twente ........................................... 47

Figure 3.9 Graph Percentage communication by email ....................... 52

Figure 5.1 Model and results of the evaluation.................................... 66

Table 2.1 Evolution IT evaluation ..................................................... 22

Table 3.1 Strategic intentions of the notebook program ..................... 32

Table 3.2 Asset pools Strategic Alignment ........................................ 34

Table 3.3 Alignment Assessment Scale ............................................. 35

Table 3.4 Results Strategic Alignment Asset pools............................ 36

Table 3.5 Internal alignment infrastructure ........................................ 37

Table 3.6 Costs of alternative configurations..................................... 39

Table 3.7 Service Level Asset pools.................................................. 43

Table 3.8 Quality Asset pools............................................................ 43

Table 3.9 Student number Eindhoven................................................ 46

Table 3.10 Student number Delft......................................................... 47

Table 3.11 Student number Twente..................................................... 47

Table 4.1 Summarized data four universities ..................................... 58

Table4.2 Summarized data four universities, incl. IT personnel........ 59

Table 4.3 Ratios PC’s per fte IT organization .................................... 60

Table 4.4 Ratios PC’s per fte IT organization, adaptation to Utrecht.. 61

Table 4.5 Budgets of the universities................................................. 62

80/85


APPENDIX C QUESTIONS AND RESULTS SURVEY COMMUNICATION

The following questions were asked at students of Eindhoven University of Technology by

means of email and interview.

General

When did you enroll in an educational program at the TU/e?

40

Do you think that there are enough PC workplaces at the campus?

highly unsatisfactory / unsatisfactory / neutral / satisfactory / highly satisfactory

Are there enough IT facilities available on the campus for students with a notebook?

highly unsatisfactory / unsatisfactory / neutral / satisfactory / highly satisfactory

81/85


Questions about communication

Which communication means do you use to contact your fellow students?

(never, incidental, weekly, daily)

- E-mail

- Instant Messenger

- Skype

- Electronic Learning Environment (Studyweb)

- Telephone

- Personal contact

Which form do you prefer to contact your fellow students?

- E-mail

- Instant Messenger

- Skype

- Electronic Learning Environment (Studyweb)

- Telephone

- Personal contact

Which communication means do you use to contact your teachers?

(never, incidental, weekly, daily)

- E-mail

- Instant Messenger

- Skype

- Electronic Learning Environment (Studyweb)

- Telephone

- Personal contact

Which form do you prefer to contact your teachers?

- E-mail

- Instant Messenger

- Skype

- Electronic Learning Environment (Studyweb)

- Telephone

- Personal contact

82/85


Propositions

highly disagree / disagree / neutral / agree / highly agree

- The possession of a notebook increases the communication with teachers more than

the use of a desktop PC (at home or in i.e. the library).

- Notebooks will play a larger role in the future for communication between student and

teacher.

- Notebooks will play a larger role in the future for communication between students.

- I am positive concerning the new ways of communication that the university is

offering me, like video conferencing and instant messaging

Open question

What are the main purposes to use the notebook?

In the tables below a summary of the answers is presented.

Number of respondents 85 General

Number Percentage

Start of study PC at home

Number Percentage yes 66 78


Communication fellow students Communication teachers

Number Percentage Number Percentage

Email

never 0 0 1 1

incidental 3 4 41 48

weekly 14 16 39 46

daily 65 76 1 1

blanc 20 24 20 24

Instant messenger

never 7 8 82 96

incidental 7 8 0 0

weekly 22 26 0 0

daily 46 54 0 0

blanc 21 25 20 24

Skype

never 60 71 81 95

incidental 16 19 1 1

weekly 3 4 0 0

daily 2 2 0 0

blanc 21 25 20 24

Electronic Learning Environment

never 25 29 25 29

incidental 21 25 32 38

weekly 15 18 22 26

daily 20 24 3 4

blanc 20 24 20 24

Telephone

Personal contact

Preference

never 6 7 68 80

incidental 21 25 12 14

weekly 23 27 2 2

daily 32 38 0 0

blanc 20 24 20 24

never 0 0 4 5

incidental 0 0 34 40

weekly 18 21 41 48

daily 64 75 3 4

blanc 20 24 20 24

Email 15 18 43 51

Messenger 5 6 1 1

ELO 0 0 3 4

Telephone 1 1 0 0

Personal contact 59 69 35 41

blanco 21 25 21 25


APPENDIX D LIST OF INTERVIEWEES

Directors of IT Service Organizations

Mrs. drs. D.C.H. Molenaars Utrecht University

P.M. van Schaik Delft University of Technology

Prof. dr. C.G.M. Sterks University of Groningen

Dr. ir. R.P. Waterham Eindhoven University of Technology

Staff members Eindhoven University of Technology

H.J. Boom IT Coordinator Industrial Design

Ing. A.C.J. Kuiper IT Coordinator Biomedical Engineering and

Mechanical Engineering

Mrs. bc. M.M.W. Kuyck-Smit Team leader Service Internal Affairs

Ir. H. v.d. Kuit Former manager notebook program

H.A.B. te Nijenhuis IT Coordinator Architecture, Building and

Planning

F.A.M. Oostrom IT Coordinator Technology Management

Ir. M.L. Raijmakers Coordinator Notebook

M.J.M. Rietjens Teacher Mathematics & Computer Science

Dr. ir. J.C. Reijenga Researcher Chemical Engineering and Chemistry

Drs. A.L. Tal IT Coordinator Applied Physics

Ing. J.H.M. Verbiesen IT Coordinator Chemical Engineering and

Chemistry

Dr. ir. T. Verhoeff Researcher Mathematics & Computer Science

Ir. M.J.M. v. Weert IT Coordinator Electrical Engineering

Ir. M. v.d. Woude IT Coordinator Mathematics & Computer Science

85/85


University of Groningen

In 1997 Eindhoven University of Technology started a notebook program. The aim of the project was to

embed IT in the education programs and so to teach the students to develop the IT skills they would

probably need in their future positions.

Ten years later an evaluation on the notebook program was performed. The research was conducted on

three aspects of the program: finances, communication between student and teacher, and competitive

advantage in comparison with the other Universities of Technology in the Netherlands.

This thesis presents the results of the evaluation based on the method New Information Economics. NIE

is developed as an ex ante evaluation method, therefore also recommendations to adapt the method for

an ex post evaluation are made.

More magazines by this user
Similar magazines