EVALUATION

AN INDEPENDENT EVALUATION OF THE LCNF
2.5 The evaluation framework
The evaluation framework is designed to be robust and credible whilst also ensuring that it
is both proportional and practicable given the information available. For both the
qualitative and quantitative assessments we have drawn on the stakeholders to help
understand the impact of the LCNF projects. As a result, the main analytical research has
been primarily drawn from responses to the questionnaires.
The assessment is designed to evaluate the LCNF in terms of both the process and
execution of the scheme, as well as on empirical evidence.
• process evaluation: we have qualitatively considered the design and implementation
of the LCNF. Understanding the effectiveness of the design and implementation is a
crucial factor in determining how reliably the outcomes of the LCNF are delivered;
and
• empirical (qualitative and quantitative) assessment: we have used qualitative and
quantitative information to test whether or not the LCNF was directly responsible for
the innovative outcomes, or whether innovation would have occurred anyway.
We have used the questionnaires described in Section 2.4.1 to ensure a wide participation
of stakeholders. We then assessed these responses to identify the key trends that would
ultimately inform our recommendations.
2.5.1 Process evaluation
This part of the evaluation considers the impact of the Ofgem procedures and process on
the outcome of the LCNF - i.e. focussing on how well the processes were designed (prior
to implementation), managed (after implementation) and delivered. The process
evaluation assessed:
• whether the qualification and evaluation criteria were appropriate and their timing of
their use in the process;
• whether information about the scheme was communicated effectively; and
• whether there was sufficient transparency in the scheme.
2.5.2 Qualitative assessment
Our qualitative assessment is driven by the feedback from key stakeholders, garnered
from the questionnaire.
The questionnaire elicited high-level, qualitative data on stakeholders’ experience of the
LCNF projects. The emphasis was not on the detail of the individual projects 21 but instead
had a broad focus, reflecting the assessment criteria set out above in Section 2.1.
21
Except where we needed to seek additional information on the individual projects and this
information was not available from the ‘LCNF learnings’ work.
PÖYRY MANAGEMENT CONSULTING
October 2016
713_Poyry_Report_Evaluation_of_the_LCNF_FINAL_Oct_2016_v700.docx
16