ForskEL and ForskVE

ForskEL and ForskVE
- a description, mapping and evaluation
of the programmes from 1998-2013

3 Introduction
4 Important findings
6 Recommendations
8 Introduction to ForskEL
and ForskVE
12 The organisation of ForskEL
and ForskVE projects
28 Network analysis of
ForskEL and ForskVE
34 Performance
39 Conclusion
The report is made by
Nadika Bulathsinhala
Postdoc - Southern University of Denmark
nab@iti.sdu.dk - October 2013
2

Introduction
The amount of public research and development investments in new
energy innovation technologies has played a crucial role in the development
and proliferation of new innovative technologies in Denmark 1 . The
public investments are especially relevant in the early stages of technology
development, where uncertainty about the innovation is greatest,
and where the privately held companies are less willing to take risks 2 .
Furthermore, research and development (R&D) in the energy sector is
often an interactive process where companies and research institutes
innovate in the interaction with each other and therefore often joined in
collaborative R&D projects. However, even though almost DKK 1 billion is
used on energy research 3 , there is not much information at project level
on how these funded projects are constructed, who participates in the
projects and information about the outcomes of the projects.
The goal of this report is threefold. First, it examines the organisational
structure of the PSO-funded (public service obligation) programmes
administrated by Energinet.dk called ForskEL and ForskVE. The aim is
to understand how these ForskEL and ForskVE projects are formed and
structured. Until now, there does not exist any general information about
ForskEL and ForskVE projects. Second, this report will give an overview
of the network structure of ForskEL and ForskVE projects, describing
how the Danish energy sector is interconnected in regard to R&D. This
will give an understanding of the complexity of the ForskEL and ForksVE
network. Finally, the report will provide the first performance evaluation
of almost hundred ForskEL and ForskVE projects. This contributes to a
deeper understanding of projects funded by Energinet.dk and what the
end-results of these projects are.
“The results of the report are based on findings from a PhD study which was conducted
from 2009-2012 and the report applies updated data from 1998-2013”.
1 Borup et al., 2009
2 Gallagher et al., 2006
3 http://energiforskning.dk/da/stats/bevilgede-tilskud-til-energiprojekter-i-2011
3

Important findings
The main challenge of the programmes is to promote R&D in
a way that strengthens the power of the private sector’s innovation
and profit incentives. The public energy programmes
can contribute to meeting future energy demands and provide
incentives for private companies to focus on R&D and collaborative
R&D projects (among various combinations of industry,
academia, and national laboratories, other governmental and
semi-governmental entities and NGOs) and provide major potential
for combining different forms of comparative advantage
in the creation of R&D and innovation.
The report is divided into three parts: the organisation of the
PSO funded projects, administrated by Energinet.dk, network
analysis of the ForskEL and ForskVE projects and performance
measurement of the projects’ output.
The organisation of ForskEL and ForskVE projects
ForskEL and ForskVE have almost allocated around DKK 2 billion
from 1998-2013. The majority of the projects are under
DKK 3 million and biomass is the energy area that has received
the largest funding amount. 56% of the projects are in the area
of applied research, but 85% of the total funding goes to demonstration
projects. The majority of the projects only has one
partner, but this percentage has declined since 2007. Furthermore,
the estimated duration time of a project is around 2.5
years. Looking at who participates in the projects, it is observed
that in 51% of the projects, a university is involved in the
projects and 16% included a research technology organisation
(RTO). It is also observed that in 61% of the projects, a privately
held company is main responsible for the project.
Network analysis
First, the report illustrates the whole ForskEL and ForskVE network
from 1998-2013. The sociogram shows that there are
three groups in the whole network: the core group, the periphery
group and the group with no relations to the others. In
other words, the sociogram shows that there are some partners
who are involved in a lot of projects and they have relations to
other partners and then are some partners who are only involved
in one project and do not have any relations. Furthermore,
when looking at the different technology phases, the network
analysis shows that the network structure changes from basic
to demonstration, indicating that partner relations changes
over time as the projects evolve.
4

Performance evaluation of ForskEL and ForskVE
When looking at project management in ForskEL and ForskVE
projects, the analysis shows that around 50% of the projects
to some extent or to a high extent met the application goals.
Furthermore, around 50% of the projects fulfilled their scheduled
activities or work packages to some or a high extent. The
analysis also shows that around 40% of the projects met their
final deadline to a low or a very low extent. Examining the projects
in regard to technological output, the analysis shows that
a majority of the projects to some or to a high extent met their
technical goals. Furthermore, the analysis shows that 60% of
the projects created knowledge that can be used in a new project
and approx. 50% of the projects created new technology.
5

Recommendations
4 Bulathsinhala, N: Innovation
processes in the energy
sector 2013
The recommendations in this report are based on the results
from the report and the doctoral dissertation 4 . The recommendations
are divided into three levels: programme level, participant
level which are the project applicants and the partners
involved in the projects and finally on expert level who are the
expert evaluators who evaluate the projects.
At programme level, the programmes should improve how it informs
the experts and participants of the objectives of the programme.
Stating very clearly what the goal of the programme
is makes it straightforward for the experts and the participants
involved in the programme. The programmes should state how
public R&D differs from private R&D, and the importance of
focus on the long-term horizon in technology development.
Moreover, it is difficult to measure R&D project performances,
when the project is publicly funded, but it is quite important to
systematic investigate what the programme and projects have
produced. This kind of performance measurement can be done
by looking at the ex-ante and ex-post processes and the programme
and project objectives.
At participant level, participants (applicants) in public R&D
projects should seriously consider why they involve themselves
in public inter-organisational R&D projects. Unmotivated
participation in R&D collaborations, because it might produce
some ’free’ information is not optimal for innovation or societal
purposes. A large amount of the electricity consumers and tax
payers’ money goes to public R&D, and the participants should
therefore consider why they apply for funding and how their
project can contribute to the energy system. This also means
that substituting private R&D with public R&D is not recommendable.
Furthermore, participants in cooperative R&D projects
are recommended to focus on explorative projects with high
risk, where they experiment and gain new knowledge. Engagement
in these kinds of projects might increase the learning of
all the participants in the project. Moreover, being involved in
inter-organisational R&D does not only require expertise in a
given field, it also requires project management. This implies
that the partners must engage in project management by giving
one partner the main responsibility of delegating tasks and
maintaining an overview of the whole project. Publically funded
6

projects with a long time span must have a project manager,
whose primary role is to manage the project and use the knowledge
created in the project. Furthermore, commitment, trust
and engagement are also necessary in inter-organisational
R&D, meaning that sleeping partners are not recommended.
When creating an inter-organisational R&D project, it can also
be beneficial for the whole project to identity and discuss each
participant’s knowledge and expertise and how they can contribute
to the project. Too many participants with the same abilities
might not contribute to making the project a success. And
finally and most importantly, the R&D projects should carefully
consider if there is a partner involved in the project, who is interested
in pushing the results to the next phase of technology
development. This type of partner, called a knowledge integrator,
can push the project to produce a successful outcome.
At expert level, it is first of all important that the experts bear in
mind that the supported technologies have to fit with the future
energy system, which may not look like the present system. It is
important that they are visionary and not too path-dependent.
Even though they are experts, they should not be too influenced
by past experience. A majority of the experts are retired employees
from the private energy sector or from an academic position.
A great amount of knowledge acquired over many years
is valuable, but it is important to bear in mind that the sector
and technology continuously changes and develops. Therefore,
it is important for the experts to be updated and open-minded.
Being visionary, it is also natural to support projects that are
risky and explorative. This demands that the experts are knowledgeable
about the technology and capable of understanding
new views on the technology. Furthermore, the experts should
also understand the context of public funding and that creating
new knowledge for the benefit of society is as valuable as
creating or improving technologies for the market. The recommendation
here is that it is essential to look beyond the private
market, without forgetting it. Finally, when evaluating projects
it is important to examine, if there is a partner (knowledge integrator)
in the projects who is motivated by taking the outcome
of the project to the next phase of technology development.
This can result in overall project success.
7

Introduction to ForskEL and ForskVE
Energinet.dk is an independent public enterprise owned by the
Danish state as represented by the Ministry of Climate, Energy
and Building. It has its own Supervisory Board. Energinet.dk
owns the natural gas transmission system, the 400 kV, 150 kV
and 132 kV electricity transmission system and is the co-owner
of the electrical interconnections to Norway, Sweden and
Germany.
The aim of ForskEL programme is to support research, development
and demonstration projects, which aims at developing
and incorporating environmentally friendly power generation
technologies including the development of an environmentally
friendly and safe electricity system. The programme mainly
prioritises projects in the areas of applied research and development
which means projects of the original character, in order
to acquire knowledge and insight towards a specific practical
aim or objective. Furthermore, projects that use knowledge to
produce new or improving existing materials, products, processes,
methods, systems or services are also highly prioritised.
The amount of funding every year is DKK 130 million.
The aim of ForskVE programme is to support projects that
promote the spread of power generation units with a smaller
power generation capacity based on renewable energy. The
amount of funding is approx. DKK 25 million.
Furthermore, the programmes ForskEL and ForskVE help to
support the energy policy objectives of security of supply, cli-
ForskEL and ForskVE are public service obligation
(PSO) programmes administered by Energinet.dk.
8

mate, environment and cost-effectiveness and they contribute
to achieving the objective of making Denmark independent of
fossil fuels in 2050.
Grants from the ForskEL and ForskVE programme can be
acquired by public or privately held companies and knowledge
institutions including universities and RTOs. Also foreign project
participants can apply for grants, but it is emphasised that
the results promote the development of the Danish electricity
system, and that the main applicant is registered by the Danish
Central Business Register.
9

Figure 1 illustrates the ForskEL and
ForskVE project evaluation process.
Step 1
Ex-ante evaluation
of project application
made by experts
Step 2
Final selection of
projects is made by
ForskEL based on the
experts’ evaluations
Step 3
Ex-post evaluation of
project outcome made
by experts
Figure 1 - The project evaluation process of ForskEL and ForskVE
10

Selection process of the projects
and the evaluation process
Different partners, such as a utility, a university and a supplier
of components can together formulate a project. The project is
often required to produce a written project application, which
describes the main aim and purpose of the project, etc.
Step one in the evaluation process is to select which projects
the programmes want to support. This is done by an ex-ante
evaluation of the project application conducted by energy
experts. At ForskEL and ForskVE, all the energy experts are divided
into different energy area sessions, where their expertise
is relevant. In the ex-ante evaluation process, at least two main
discussants evaluate each project application and fill in a form
provided by Energinet.dk. The form has formulated different
measures from how good is the description of the technology
to does the project have value for money and impact? It is mandatory
for the discussants, after the points have been awarded,
to state the reasons for their evaluation approach to the written
format and make an argument for their evaluation.
8-14 experts are represented in an evaluation session, depending
on the technology, and the discussants present their evaluation
form on a projector, enabling everyone to see which
score each project has received and why. After the discussants’
presentation, the rest of the experts can ask questions to the
discussants, and the forum is open for discussion between the
experts. If particular experts are incompetent because of their
own interests in a given project, they are excluded from the
evaluation of the project application.
The aim of the discussion is to find a consensus score that
covers all the experts’ opinions, and all the comments are
written down by the programme research coordinator. After
the main evaluation of the project applications, step two is
the crucial selection of projects that will receive funding. This
is done by research coordinators from Energinet.dk. Step
three, an ex-post evaluation of the finished projects, is made
by experts.
The duration time between step two and step three can be
around three to four years, which means that it is not always
the same experts that make the ex-ante evaluation in step one
and the ex-post evaluation of the project outcome in step three.
11

The organisation of ForskEL
and ForskVE projects
Over the years, ForskEL and ForskVE have prioritised different
focus areas in their call depending on the overall political
energy strategy, and the annual call and the prioritisation is
approved by the Minister of Climate and Energy. This implies
that one year, the majority of the focus might be on biomass
and over time, the focus might change and be on solar power.
Moreover, some energy areas are only represented some
years and then they are faded out such as biofuel. A reason for
this can be that another energy programme prioritises biofuel
or that the present technology has become commercial. Other
projects such as non-environmental fossil energy projects do
not receive public funding anymore, because it does not fit with
the aim of the programme which is to support environmental
energy technologies.
A description of the energy areas
over the years and funding
The section describes all energy areas that have received funding
from ForskEL and ForskVE since 1998-2013, and all the
supported energy areas have to have a relation to the energy
system. The purpose of the description is to give a basic understanding
of the technologies supported by the programmes and
describe the energy categories explained in figure 2.
Figure 2 illustrates the number of projects in each energy area
that has received funding from 1998-2013. It is shown that in
1999, a majority of the projects were in the area of biomass,
but the support has declined since 2010. The figure also illustrates
that new energy areas have evolved like Electricity
storage and Demand and response. Furthermore, the figure
shows how the support to the different energy areas differs
over time. A reason for the changes can be political agendas.
For example, the Danish government wanted to focus on a certain
technology such as hydrogen power in the beginning of the
2000s and therefore, the area was allocated a great deal of
financial funding. Another reason for the decline of a technology
can be the evolvement of the technology. Concurrently with
a technology becoming more commercial, it may receive less
public funding, as the need for public funding reduces as the
technology becomes more mature.
12

Number of projects
30
25
20
15
10
5
Figure 2 - ForskEL and ForskVE projects from 1998-2013
Number of projects
30
0
1998
1999
Biofuel
2000
2001
2002
Hydrogen
2003
2004
2005
Solar
2006
2007
2008
Control and demand
2009
2010
2011
25
Biomass
Wave
Smart grid
Fossil
Wind
Other
Demand and response
Electricity storrage
20
15
10
5
0
1998
1999
2000
2001
2002
2003
2004
2005
2006
2007
2008
2009
2010
2011
2012
2013
Biofuel
Hydrogen
Solar
Control and demand
Biomass
Wave
Smart grid
Fossil
Wind
Other
Demand and response
“The figure above illustrates all the ForskEL and ForskVE projects from 1998-2013.
The dataset consist of 562 projects and all the energy areas are represented”.
Electricity storrage
13

Description of energy areas supported by ForskEL and ForskVE
Biofuel is a fuel that contains energy from geologically recent carbon fixation. Biofuel is produced from living organisms and
examples of this carbon fixation occur in plants and microalgae. The use of biofuel is mostly seen in the transportation sector.
Hydrogen is a zero-emission fuel which uses electrochemical cells or combustion in internal engines, to power vehicles and electric
devices. A fuel cell combines hydrogen and oxygen to produce electricity, heat, and water. Fuel cells are often compared to batteries.
Solar energy technologies use the sun’s energy and light to provide heat and electricity for homes, businesses, and industry. There
are different solar energy technologies like photovoltaic systems and concentrated solar power (CSP) that produce electricity directly
from the sun light or solar heating that heats water with sun light radiation.
Biomass is the use of biomass to generate electricity. There are six major types of bio power systems which are direct-fired, co-firing,
gasification, anaerobic digestion, pyrolysis and small, modular. Most of the biomass plants in the world use direct-fired systems,
where they burn bioenergy feedstock directly to produce steam.
Smart grid is a modernised electrical grid that uses information and communications technology to gather and act on information,
such as information about the behaviours of suppliers and consumers. The goal is to improve the efficiency, reliability, economics, and
sustainability of the production and distribution of electricity.
Wind turbines are mounted on a tower to capture as much energy as possible. At 100 feet (30 metres) or more aboveground,
they can take advantage of the faster and less turbulent wind. Wind turbines can be used as stand-alone applications, or they can be
connected to a utility power grid or even combined with a photovoltaic (solar cell) system.
14

Wave power devices extract energy from the surface motion of ocean waves or from pressure fluctuations below the surface.
Machinery able to exploit wave power is generally known as a wave energy converter.
Fossil fuels formed by natural processes such as anaerobic decomposition of buried dead organisms. Fossil fuels contain high
percentages of carbon and include coal, petroleum, and natural gas. ForskEL and ForskVE do not in the present time support projects
in the area of fossil energy.
Control and demand and Demand and response are two areas that support the creation of what Smart Grid requires and
coordinated development and demonstration activities that promote the maturation of technologies and the solutions that must be the
building blocks of the future intelligent electricity system.
Electricity storage is accomplished by devices or physical media that store energy to perform useful operation at a later time. A
device that stores energy is sometimes called an accumulator. Some technologies provide only short-term energy storage, and others
can be very long-term such as power to gas using hydrogen or methane and the storage of heat or cold between opposing seasons in
deep aquifers or bedrock.
Other is technologies that are currently at the research stage and are far from being commercial. Some of these technologies are
relevant for environmentally friendly power generation and integration; they can be supported, to the extent that it is applied research,
feasibility studies or pilot projects.
15

6 - 8.9 mio. DKK.
3 - 5.9 mio. DKK.
PSO Funding
Figure 3 illustrates the number of projects and the size of the
The total amount of financial funding from the different PSO
total budgets divided into four groups. It shows that 63% of the
programmes affiliated with Energinet.dk from 1998-2013 is
projects have received a funding amount smaller than DKK 3
DKK 1,867,425,664. Every year, the programmes have a total
million. A reason for this is that many of the projects are fea-
of approx. DKK 150 million which they can grant qualified pro-
sibility studies where the aim is to investigate, whether there
jects. If the programmes do not utilise the whole budget, it is
is a valid foundation for a bigger project in a given energy area
possible to assign the rest of the funding to the next year. The
and whether the new technological possibility is a realistic op-
figure below shows the total amount of funding from 1998-
portunity.
2013 divided on project size.
Furthermore, the second largest group with 22% is the group
Figure 3
The total amount
of funding divided
on project size
Number of projects
400
352
300
of projects in the budget area of DKK 3-6 million and the third
largest group is projects in the budget area of DKK 6-9 million.
The last group of projects with a budget from DKK 9 million and
above is only 7%, indicating that not many projects receive a
relatively large amount of PSO funding. However, it is observed
that there is an increase in the projects with a budget over
200
DKK 9 million since 2009. A reason for this can be that since
2009, the programmes have increased their focus on bigger
124
consortium projects that demands a larger budget. The aim of
100
the consortium projects is to support larger projects that inclu-
0
44 40
de different partners from all the technological development
phases. This means that the aim is to support projects where
the partners can execute different tasks in the technological
16
9 mio. DKK. and over
Under 3 mio.DKK.
development phases from applied research to demonstration/
commercialisation.

When examining how the total amount of PSO funding is divided
between the different energy areas, it gives an interesting
picture. Over the years, some energy areas have received more
financial support than others.
The area of biomass has received 26% of the total amount of
PSO funding, illustrating that even though projects in biomass
have declined over the years (see figure 2), this energy area
has over a period of time collected the largest amount of the
total PSO funding.
The second largest energy area that has been allocated a great
deal of funding is hydrogen (23%). The relatively new funding
area called Demand and response has received around 5% of
the total PSO funding, indicating that this is an area that now
has a lot of focus. ForskEL has prioritised this area due to the
great attention to and implementation of Smart grid technology.
An energy area which has received less funding is biofuel. A
reason for this is that biofuel is more relevant in the transportation
sector.
What is important to emphasize is that the whole picture of
funding allocation can change when looking at the total number
of projects and funding. Table 1 illustrates the total number of
projects divided into energy areas from 1998-2013.
The table illustrates that even though the energy area of hydrogen
has received a large of the portion of the total amount
of PSO funding, the number of hydrogen projects is relatively
small compared to biomass or wind.
“An energy area which has received less funding is biofuel. A reason for
this is that biofuel is more relevant in the transportation sector”.
18

Figure 4 - Energy area divided on the total amount of PSO funding from 1998-2013
PSO funding (mio. DKK)
Table 1 - Total number of projects
divided into energy areas from 1998-2013
600
Energy area
Number of projects
Biofuel 20
500
Biomass 173
Wave 34
400
Hydrogen 69
Smart grid 34
300
Fossil 39
Solar 59
200
Wind 78
Control and demand 17
100
Demand and response 20
Electricity storage 3
0
Other 16
Total 562
Biofuel
Biomass
Wave
Hydrogen
Smart grid
Fossil
Solar
Wind
Other
Control and demand
Demand and response
Electricity storrage
19

the technological development phases
The technological development phases (basic research, applied
research, demonstration, and commercialisation) are in reality
not a linear process, but a muddle of overlapping processes
and feedback. The ’muddle’ includes partnerships and interactions
within and between sectors (government, industry, academia,
NGOs) and across national boundaries, eg in the EU.
One of the two main reasons why the energy industry does R&D
is that such activities foster the development of new products
or improve existing products and the companies’ own energy
usage (and allow them to meet regulations); the second reason
is the ultimate intention, which is to stay in business and maintain
or increase the competitive position in the market.
Figure 5 below illustrates the different phases in technology
development. One project might start in basic research doing
pure science and after some years, depending on the project’s
complexity and funding, proceed to the next phase if the results
are applicable. In applied research, the goal is to turn the
pure science into a more strategic outcome, where the results
over time can become a commercial output. The demonstration
phase tests the R&D in approximating real-world conditions
and scales up the technology to demonstrate real world feasibility.
Finally, if the technical feasibility of a new technology is
a success, the next phase is to introduce the technology to the
market. From the demonstration and commercialisation phase,
there is a feedback loop back to applied research via testing
Figure 5 - The technological development process
Basic research
Applied research
Demonstration
Commercialization
Discoveries
Academic
curiousity
Applied and
strategic R&D
Up-scaling R&D
and testing
Price reduction
Competitive
technology
Feedback
Technology push
20

and up-scaling of the technology. Furthermore, the figure
shows that there is a technology push by politically supporting
collaborative R&D projects that meet the overall energy policy.
The projects that apply for funding from ForskEL and ForskVE
have over time been in different phases in the technology development
process, from basic research to demonstration. Figure
6 shows how the total amount of PSO funding is divided
between phases. The focus of the programmes ForskEL and
ForskVE is now predominantly on applied research and demonstration
and therefore not on basic research, as Figure 5 above
illustrates. Only around 3% of the total amount of funding from
1998-2013 is allocated to basic research. 12% of the total PSO
funding is allocated to applied research.
Figure 6 - Total PSO budget and technology phases
3%
12%
85%
Basic
Applied research
Demonstration
Figure 6 also illustrates that the majority of the total PSO funding
goes to demonstration. A reason for this is that demonstration
projects are financially demanding, because the aim of
demonstration projects is to scale up the facilities and test a
technology. However, 56% (not shown in the figure) of all projects
are in the technology area of applied research, indicating
that the programme prioritises projects primarily in technology
phase of applied research. Only 29% (not shown in the figure)
of the projects are in the technology phase of demonstration,
even though they financially constitute 85% of the total amount
of PSO funding.
21

The construction of the projects
It is important to describe the composition and construction of
the ForskEL and ForskVE projects because this gives a better
understanding of the projects in general and information about
how the projects that have received funding from 1998 to 2013
are constructed.
As mentioned before, ForskEL or ForskVE projects can involve
different partners who have a mutual interest in collaborating
on a technological challenge, e.g. a project in the area of solar
energy might involve a university with the latest knowledge
about solar energy technology, a supplier of components that
can be used in the creation of the new technology and an energy
company that in the end might be interested in implementing
the technology. Figure 7 illustrates a possible structure of
a ForskEL or ForskVE project.
Figure 7 - Example of
a collaborative R&D
project constellation
University
The average of the total number of partners in projects is approx.
three partners. Figure 8 illustrates that 29% of the projects
only have one partner in the project. These projects are
mainly in the technology phases of applied research (27.5%)
and demonstration (32.9%). However, since 2007 projects
with only one partner has declined purposely due to the increased
focus on the triple helix concept. The goal of the triple
helix concept is to increase innovation and the development of
new knowledge through the collaboration between university,
Supplier of
components
Energy
company
22

industry and government and intersection of the three institutional
spheres. Furthermore, from around 2010, projects with
more than four partners have slightly increased, indicating that
projects have become larger over the years, when looking at
the number of partners.
Figure 9 below illustrates the estimated duration of the projects.
The average of the estimated duration time of a project
is around 2.5 years. When a project applies for funding, they
also have to estimate a termination date for the project in their
application.
In addition, when examining the budget it is also observed
that since 2005, the number of projects receiving funding has
decreased, indicating that ForskEL and ForskVE now support
fewer project with larger budgets including more partners than
before.
The figure shows that around 32% of the projects have an estimated
duration of approx. three years and 23% of the projects
have estimated a duration time of approx. four years. Only
around 19% of the projects have a longer duration time longer
than four years and approx. 54% of these projects are in the
technology phase of applied research. It is also observed that
since 2003, the duration time of the projects has decreased.
Figure 8 - Number of partners in the projects
Figure 9 - Project duration
Number of projects
Number of projects
180
160
140
120
100
80
60
40
20
160
140
120
100
80
60
40
20
0
1 2 3 4 5 6 7 8 9 10 11 12 13 14
0
1 2 3 4 5 6 7 8 9
23

Who participates in the projects?
ForskEL and ForskVE encourage everybody with an interest in
developing environmental technologies that can be implemented
in the electricity system to submit a project application.
Figure 10 illustrates that around 51% of the projects have a
university included in the project. As expected, a great deal of
projects in the area of applied research have included a university,
but it is also important to observe that the presence of
universities is identified in all the technology phases, indicating
that universities do play a significant role in energy research.
A RTO is often a privately held company which specialises in
consulting in research and technology. In Denmark they are
called advanced technology groups (GTS) and there are nine
independent organisations across the country specialised in
different areas. Figure 11 illustrates that 16% of the projects
includes an RTO and they are primarily in the technology phases
applied research and demonstration (not shown in the figure).
International partners can be a foreign university or a privately
held company. Figure 12 shows the percentage of international
partners in the projects of ForskEL and ForskVE. Around
11% of the projects include international partners and in nearly
57% of the projects which includes an international partner, the
partner is a university (not shown in the figure).
It is also observed that the number of international partners is
relatively constant over the years. The only requirement of a
project with an international partner is that the main responsible
partner of the project is a Danish institution. Furthermore,
ForskEL and ForskVE support Danish partners in international
projects such as ERA-Net projects in Smart grid, Biomass and
Solar energy. These projects are not included in the dataset as
projects with an international partner.
Being included in a project and being responsible for a project
is two different things. Being the main responsible partner of a
project implies that the current partner has the overall responsibility
for the project and they often also invest more time and
money in the project compared to the other partners depending
on the arrangement of the project. Figure 13 illustrates the responsible
partners in the funded projects, divided into different
groups such as universities, private companies, RTO and public
institutions such as a municipality.
The figure shows that a majority of the projects have a privately
held company as the main responsible for the projects. This
means that it for example is a utility or a producer of electricity
that forms a project, incorporating others partners in the project.
Moreover, it is observed that the majority of these projects
are in the technology phases applied research and demonstration.
24

Figure 10 - The presence of universities in the projects
51%
Figure 11 - The presence of a RTO in the projects
16%
51%
Projects including a
university
16%
Projects
including a RTO
49%
49%
Figure 12 - The presence of an international
partner in the projects
89%
89%
11%
11%
Projects not including
a universaty
Projects including a
university
Projects not including
a universaty
Projects not including
an international partner
Projects including an
international partner
Projects not including
an international partner
Projects including an
international partner
84%
84%
65%
65%
8% 1%
Figure 13 - Partners responsible for
the projects divided into groups
8% 1%
26%
26%
Projects not
including
Projects
a RTO
including a RTO
Projects not
including a RTO
University
Private company
RTO
Public University institution
Private company
RTO
Public institution
25

The second largest group is universities. They are mainly responsible
for project in basic research and applied research.
Finally, RTOs are the second smallest group with 8% and they
are mostly represented in applied research confirming their role
as the mediator between universities and private companies.
Energy areas and the technology
development phases
It is interesting to examine where the different energy areas are
represented in the technological development phases, because
it is relevant to illustrate the position of different technologies
and compare them. Table 2 shows the projects divided into
phases and energy areas. The table does not illustrate the development
over time, because it only sums up all the projects
from 1998-2013. However, it can be concluded that the table
does show some kind of progress, especially with the technologies
that are represented through all the technological development
phases such as biomass.
All energy areas besides Biofuel, Control and regulation, Demand
and regulation and Electricity storage are represented
in all the technology phases. This indicates the possibility of
some of the supported projects from ForskEL and ForskVE
have progressed through time. However, since 2009 there has
been a drastic decrease of projects in basic research supported
by ForskEL and ForskVE. A reason for this can be that all
the energy programmes have made a more clearly division of
the different phases in technological development between the
programmes, meaning that some of the programmes have their
primary focus on basic research and others in demonstration.
ForskEL and ForskVE are primarily in the area of applied research
and demonstration.
Furthermore, as expected, table 3 shows that the privately
held companies primarily are represented in applied research
and demonstration. However, the table also shows that private
companies were engaged in basic research. A reason for this
can be that private companies in the energy sector have employees
that have the technological expertise and competences to
engage in projects in the area of basic research. The table also
shows as expected that the involvement of research institutions
decreases through the phases.
26

Table 2 - Number of projects divided into energy areas and technology phases
research Applied research Demonstration
Biofuel 0 14 6
Control and regulation 0 9 8
Demand and regulation 0 7 12
Electricity storage 0 2 2
Biomass 26 106 36
Wave 1 20 14
Hydrogen 17 36 12
Smart grid 8 15 10
Fossil 5 15 19
Solar 4 25 28
Wind 14 50 10
Other 6 6 2
Table 3 - Private and research institutions divided into technology phases
research Applied research Demonstration
A private company is included in R&D projects 51 281 148
Only research institutions are included in the R&D project 30 24 11
27

Network analysis
of ForskEL and ForskVE
Social network analysis (SNA) is an approach that can be used
to observe and identify relationships in networks through advanced
calculations. The aim of this section is to give an overall
picture of the network of ForskEL and ForskVE projects and
how the participants (partners) are related in the whole network
and divided into the different phases of technology development.
Furthermore, the SNA can be used to identify if there
are participants, who are not included in the main network and
if there are participants who have many relations to other actors
in the sector.
The SNA sociogram consists of nodes which represent individual
partners within the network and ties which represent
relationships between the partners in a network. This report
will depict the network of ForskEL and ForskVE projects by
using social network sociograms, where nodes are represented
as points and ties are represented as lines. It is important to
bear in mind that all the sociograms in the report have a time
dimension from 1998-2013 and are an aggregated view of the
network.
The whole ForskEL and ForskVE network
When examining R&D in the Danish energy sector, it is observed
that the sector is filled with a great amount of different
participants from privately held companies such as producers
of electricity to research institutions such as universities.
The sociogram illustrates all the participants in ForskEL and
ForskVE projects from 1998-2013. The white nodes are the
different participants and the larger the nodes are, the more
a participant is involved in different projects. The blue nodes
are projects, and if a participant is involved in many projects, a
group of blue nodes are seen near a big white node.
From the sociogram, three main groups are identified. There is
a core group of participants marked with a red circle, who are
involved in many projects and have numerous relations to other
participants. Then there is another group marked with the green
circle and they are outside the main core of the network, but
they are also involved in projects and have relations to other
participants in the sector. Finally, the third group is outside the
28

green circle and a majority of these nodes do not have a relationship
to other participants in the network. Questions that
are interesting to ask are whether the participants outside the
green circle have access to other participants’ experiences and
knowledge or whether participants in the red circle are members
of an exclusive group only sharing knowledge within the
small network. Furthermore, projects not being a part of the
larger network might indicate that ForskEL and ForskVE dare
to support newcomers to the network and not always support
familiar participants with a known network.
There are three participants that over time have 102, 91 and 68
different relations. These participants are two Danish universities
and a former producer of electricity.
Figure 14
Sociogram of the whole ForskEL
and ForskVE network and projects
29

Technology development phases
and the network
The technology development phases such as basic, applied and
demonstration have previously been described in this report.
The aim of this is to illustrate the network in the different phases
and show how the network changes, the more the projects
involve.
During the technology development phase of basic research
(figure 15), it is observed that there are some large-scale participants
(white nodes) that are involved in a great amount of
projects (blue nodes). There are 146 nodes and 200 relations
in this network. The most active participant in basic research
is involved in 43 projects.
What it important to show is that within basic research, the network
has a close interaction between the projects and participants
which means that the distance between the participants
is relatively small and there is a strong collaboration culture.
Figure 15 - The ForskEL and ForskVE
network in basic research
30

In applied research (figure 16) the network is completely different
compared to the network in basic research. There are 738
nodes and 939 relations in this network and the most active
participant is involved in 64 projects. This phase has the most
participants compared to the other phases; however, more
participants in a network can also affect the closeness of the
network because of the bigger spread.
Figure 16 - The ForskEL and ForskVE
network in applied research
31

The last technology development phase is the demonstration
phase. Here 480 nodes are observed and there are 503 relations
between the nodes and the most active participant is
involved in 21 projects. The demonstration phase differs from
the other phases by being the most fragmented network, indicating
that this network is highly specialised.
To sum up, by applying the SNA it has been possible to identify
the network structure and provide information about the whole
network of ForskEL and ForskVE and how the energy sector
in regard to R&D is connected. It has shown that over time, a
majority of the different participants are involved in different
projects and have relations to one another.
32

Figure 17
The ForskEL and
ForskVE network
in demonstration
33

Performance
The definition of performance is the accomplishment of a given
task measured against present known standards of accuracy,
completeness, cost and speed. The data in this section consists
of 135 end reports, carried out by 27 different evaluators.
20 evaluators agreed to participate in the survey, which resulted
in a response rate of 74.0%.
In the process, the end report of the expert evaluator and a
copy of the survey for each report were sent to the expert evaluators
by email. The evaluator was requested to evaluate the
end reports again and then fill in a questionnaire for each project
they had previously evaluated. 105 surveys were returned,
resulting in a response rate of 77.7%.
investigates the short-term performance. A future project will
look more into the long-term outcome of ForskEL and ForskVE
projects.
A way to examine performance on a short term is to investigate
if the projects fulfil the obligations written in the project application
and if the final results of the project are as the project
aimed for. But bear in mind that measuring performance on a
short and long term can be complex especially in public supported
projects. Overall, the level of risk in public supported
projects can be relatively high, meaning that a great deal of
the projects may fail and some projects may succeed in project
management but fail in technology output and vice versa.
When examining ForskEL and ForskVE projects, it is relevant
to investigate, when looking at the short term perspective,
the project application and output. It is also relevant to examine
the project output in a long-term perspective such as five
years after the project has ended, in order to see whether the
technology is implemented and deployed, but this report only
Therefore, the analysis will start with performance measurements
on project management examining how well the projects
manage the project. Subsequently, the analysis will go more
into the technological output and study how relevant the final
results of the projects are.
34

As a final point, investigating performance of public supported
projects is not a simple task. As mentioned before, the programmes
support projects that have more than a potential economic
dimension. The projects also have to contain a societal
and political dimension; therefore, it is important to have this
complexity in mind.
project management
Project management is the discipline of planning, organising,
motivating and controlling resources to achieve specific goals.
By employing good project management, it can be less challenging
to meet the goals and milestones of the project. Table
4 shows to what extent the projects lived up to the goals in the
project application.
The table shows that around 50% of the projects to some
extent or to a high extent lived up to the application goals. This
indicates that around half of the projects met their application
goals as described in the application. In a project, the applicants
have to describe different activities and work packages.
Table 5 below illustrates to what extent the projects fulfilled the
scheduled activities and work packages.
Table 4: Goals in the project application
To a high In some To a To a very
extent extent Neutral low extent low extent Total
To what extent
did the project live
up to the project
application goals? 24 (23%) 28 (27%) 32 (31%) 14 (13%) 6 (6%) 104
Table 5 - Activities and work packages
To a high In some To a To a very
extent extent Neutral low extent low extent Total
To what extent did
the project fulfil the
scheduled activities
or work packages? 14 (14%) 39 (38%) 30 (29%) 15 (15%) 4 (4%) 102
35

Table 5 shows that around 50% of the projects to some or a
high extent fulfilled their scheduled activities or work packages.
Only 4% of the projects fulfilled their activities or work
packages to a very low extent.
In the application, the project applicants have to state when
they plan to finalise the project. There is a deadline and they
have to write an end report that includes their results and conclusion.
The table below illustrates if the project met its final
deadline and to what extent.
Table 6 shows that around 40% of the projects met their final
deadline to a low or a very low extent. This means that a great
deal of the projects are not capable of finishing the projects
according to the scheduled time. There are a lot of different
reasons for why the projects are delayed. A reason can be that
a supplier of components could not deliver on time, which affects
the rest of the process or it can be a change of partners
in the projects or simply new knowledge that might affect the
final outcome. Furthermore, it can also be due to poor project
management that the projects have a hard time meeting their
deadlines.
The analysis also shows that 62% (not shown in the table) of
the projects when looking at the overall project management
was a success, indicating that even though the projects might
not meet the final deadline, the overall project management in
most of the projects is sufficient.
Technological output
Technology output means the final results or findings of the
project. The majority of the projects aim at creating new technologies
or innovations and the following analysis will illustrate
how the projects managed to do so.
All projects have to state in their application what their technical
goals are. Some projects aim at increasing the efficiency of
an existing technology, while others try to develop and test a
new technology. The table below illustrates to what extent the
projects met their technical goals.
Table 6 - Meeting final deadline
To a high extent In some extent Neutral To a low extent To a very low extent Total
To what extent did the
project meet its final deadline? 4 (4%) 25 (27%) 27 (29%) 22 (23%) 16 (17%) 94
36

Table 7 shows that a majority of the projects to some or to a
high extent met their technical goals. Only 5%of the projects
met their technical goals to a very low extent. What is interesting
to see is that 50% of the projects that met their final
deadlines to a low extent did, met their technical goals to some
extent (not shown in the table). This indicates that there can be
a relationship between not meeting final deadlines and achieving
the technical goals of the project.
Projects from ForskEL and ForskVE often have a relatively long
time horizon. As previously explained in this report, projects
can go through different technological phases and it can take
years before a project becomes commercial. Therefore, the
goal of a project is not always to create a commercial product,
but it can also be to create some new knowledge that can be
used in a new project, going from applied research to demonstration.
Tabel 8 illustrates to what extent the projects created
knowledge that could be used in a new project.
Table 8 shows that 60% of the projects did create knowledge
that can be used in a new project. This indicates that projects
in the technology phases of either basic, applied or demonstration
created some knowledge that might be useful in a new
project. However, newly created knowledge does not also have
to be positive results. It can also be negative results that can
contribute with the knowledge of what does not work and this
knowledge can be used in a new project.
Table 7 - Projects and meeting technical goals
To a high In some To a To a very
extent extent Neutral low extent low extent Total
To what extent did
the project fulfil its
technical goals? 12 (12%) 40 (38%) 29 (28%) 18 (17%) 5 (5%) 104
Table 8 - Projects and new created knowledge
To a high In some To a To a very
extent extent Neutral low extent low extent Total
To what extent did
the project create
knowledge that
can be used in
a new project? 20 (19%) 43 (41%) 22 (21%) 14 (13%) 6 (6%) 105
37

One of the main aims of ForskEL and ForskVE is to support
projects that develop new technology. Developing a technology
can be very costly and time demanding. For many private
companies, it can be very risky and cost-intensive to focus on
the development of new technologies; therefore, the programmes
prioritise to support projects with the aim of developing
a new technology. The table below illustrates to what extent
the project supported by ForskEL and ForskVE created a new
technology.
Table 9 shows that around 50% of the projects create a new
technology, meaning that half of the projects did not create a
new technology. A reason might be that it takes a long time to
develop a technology and that a great deal of the projects can
be in the area of incremental innovation, where the goal is to
refine the existing technology and to test them. However, this
might also indicate that half of the projects might have failed in
developing a new technology and they did not create the output
that they have aimed for.
Finally, asking the expert evaluators about whether the overall
technical results of the projects were a success, only 52% answered
yes. Additionally, 48% of the projects produced scientific
publications signifies that new knowledge was created, but
only 10% of projects devised patent applications. Furthermore,
only 11% of the projects produced a PhD. project (not shown
in the table).
Table 9 - Projects and new technology
To a high extent In some extent Neutral To a low extent To a very low extent Total
To what extent is the
technology new? 8 (8%) 42 (41%) 35 (34%) 13 (12%) 5 (5%) 103
38

Conclusion
The goal of this report was threefold. First, it presented and
then examined the organisational structure of the ForskEL and
ForskVE projects. Until now there does not exist any general
information about ForskEL and ForskVE. Second, this report
gave an overview of the network structure of ForskEL and
ForskVE projects, describing how the Danish energy sector is
interconnected in regard to R&D. This has also not been done
before and this gave knowledge about the ForskEL and Forsk-
VE network. Finally, the report provided the first performance
evaluation of almost hundred ForskEL and ForskVE projects.
This has also not been done before and the results are interesting
for future project evaluation, because it contributes with
deeper understanding of projects funded by Energinet.dk and
what these projects have produced on the short term.
In the future, there is a need for a larger performance evaluation
that examines the outcome of the projects on the short and
long term. This can give a better understanding to ForskEL and
ForskVE of which projects to give financial support, what the
projects need to fulfil their goals, and finally how to increase
the level of success of the final output.
I would like to thank Energinet.dk, all the research coordinators
at ForskEL and ForskVE, Jens Martinus Pedersen, Knut Berge,
Niels Laursen, Per Kristensen, Tue Hald, Jan Vedde, Kurt S.
Hansen, Bjarne Maribo Pedersen, Lars Nikolaisen, Ole Kristensen,
Kaj Isaksen, Lasse Rosendahl, Viktor Jensen, Fritz Luxhøj,
Thomas Astrup, Krister Ståhl, Peter Ahm and Ole Jess Olsen for
insightful comments and assistance.
39