Evaluation of Railway Projects in the European Union

More documents

Recommendations

Info

event, a more conservative approach would have been preferable at least in the substantial number ofrepeat projects with a history of overestimates.Project implementation schedule. As mentioned before about two thirds of the projects evaluated hadimplementation delays, one third of which by three or more years. This suggests that the Bank shouldanalyze the potential causes of such delays more systematically and propose measures to minimizethem where possible. Such an analysis could be a straightforward exercise, for instance in the form ofa checklist, because most potential causes of delay are known, i.e. lack of agreement withlocal/regional authorities on project details affecting them 23 ; incomplete project designs andimplementation plans; political interference in project implementation; natural causes such asgeological problems or archaeological finds. Although experience clearly shows that the Bank’sinfluence is limited without the promoter’s cooperation even when it has identified the problems, suchan analysis would at least result in a more realistic assessment of a project’s strengths and weaknesses.Economic analysis. All projects included a quantified economic analysis, albeit in varying detail,completeness and quality. For example, in one appraisal the economic life of the project wasoverstated and in another a different method (residual value) was used. In other projects, operatingand maintenance costs were not taken into account or the economic viability was based on the EIRRestimate of a much broader investment programme and not the project as such. Other economicanalyses omitted the standard practice of sensitivity analysis as a gauge of the potential project risk.The end result was that only a few ex ante estimates of the economic viability of the projects evaluatedin depth were methodologically entirely sound.Where possible, the ex post evaluation attempted to estimate the project EIRRs by using the standardmethodology but this could not always be achieved because the information was not available or notprovided for reasons of confidentiality (despite the Promoters’ contractual commitment to providesuch information). Obviously, the EIRRs calculated for projects that are delayed are subject to changeonce those projects have been completed and are operational. However, that does not alter theconclusion that their economic rates of return were much lower than estimated at appraisal because ofthe substantial delays, cost increases already incurred and lower demand, not to mention theoccasional weaknesses sometimes in the appraisal methodology.Risk Analysis. The Bank’s analysis of the credit risks that it may incur with a project has generallybeen satisfactory. As regards such other risks as traffic risk, implementation delays and cost increases,the over-optimism of the appraisal teams has already been mentioned. In some appraisals these riskswere taken into account by means of a sensitivity analysis, in others not. For example, the appraisal ofa follow-on project had identified a high risk of cost increases and delays as a result of the experiencewith the first project, but no sensitivity analysis was undertaken to quantify the impact of those factorson the project’s viability. In general, it can be said that even where sensitivity analyses wereperformed the worst-case hypothesis usually proved too optimistic in the ex post evaluation.Apparently it is difficult to think the “unthinkable”, i.e. that a project might fail. An antidote could bethe inclusion of a null hypothesis in the sensitivity analysis, which would examine under whichconditions the proposed project would fail and give an estimate of the likelihood of such failure.Even when the risks were properly identified, mitigation measures were rarely recommended oraccepted by promoters. For instance, in one (very large) project the promoter hired an expert at theBank’s recommendation to monitor costs and project implementation, with the result that the Bankhad an excellent understanding of the reasons for the delays and cost increases but no influence ontheir mitigation. A further Bank recommendation on how to deal with cost overruns and delays wasnot accepted by the promoter. A similar situation occurred in another large project, where costoverruns and delays had already been experienced during the initial implementation stage. The Bankproposed assisting the promoter with mitigation measures but this was rejected. The project has nowbeen delayed by three years. Of course, it is not possible to conclude whether acceptance of the23There was only one project that was on budget and on time where potential problems with local andregional authorities were discussed and cleared before project implementation started.21
Bank’s advice would have made any difference. The lesson that can be learned from this experience isthat the Bank has little leverage to influence the outcome of a project one way or the other, unless thepromoter takes ownership of the Bank’s recommendations.6.5 Financing arrangementsThe financing arrangements for 18 of the projects reviewed (in depth and desk reviews) consisted ofdirect loans to the Promoters. In 11 projects, financing was arranged via an intermediary, most often aGovernment agency or a special purpose vehicle created for the operation, and in one case morecomplex arrangements in the form of public private partnerships were chosen. In one of these cases aspecialist was called in (at the expense of the Promoter) to assist with the design of the financialinstrument.6.6 MonitoringThe Bank’s appraisal team normally recommends the type of project monitoring that it deemsappropriate in the context of the complexity and potential risks of the project concerned and in thelight of its knowledge of the borrower’s implementation arrangements and technical competence.Accordingly, monitoring categories 1 to 3 were assigned, with increasing degrees of monitoring. Asregards the projects evaluated in depth and via desk studies the following monitoring categories wereassigned: category 1, five projects; category 2, twelve projects; category 3, ten projects. Six projectswere without a monitoring category.The actual monitoring was carried out in the majority of projects less in accordance with the assignedcategory than in accordance with the actual difficulties encountered. This has the benefit of utilizingthe Bank’s scarce staff resources in an appropriate way. Thus, projects whose implementation wassmooth were not monitored at all. On the other hand, problem projects were monitored intensely,including in one case the utilization of a specialist. In other projects, intense monitoring was a byproductof follow-on operations by the Bank.However, the ultimate objectives of the monitoring exercises were not clear. Thus, even in the caseswhere the Bank’s monitoring identified serious issues, there was no concrete action on the part of theBank to mitigate or eliminate those issues, unless the Bank’s loans were at risk directly. Thishappened, for instance, in one project, when the guarantees for the Bank’s loans became worthless andnew guarantee structures had to be set up. It can therefore only be concluded that monitoring was aresource-driven process, with as its main purpose at best to keep the Bank informed of a project’sdifficulties, and that the Bank does not enter into negotiation as long as its own loans are not at risk.On average, less than 3 progress reports were produced for each of the 29 projects evaluated.Nevertheless, important modifications in certain projects were discovered too late. For instance, oneproject was expanded geographically by 50% and the equipment to be procured also changedsubstantively. In another project, only 340 project components out of the original 820 were financed,while 389 new components were added. These changes should at least have warranted a reappraisal ofthe Bank’s involvement, in order to ensure that the project was still viable and its objectives were stillvalid (again these two projects are very large projects). Also the direct impact on the Bank’s accountsis low, the promotion of the Bank’s “expertise” is not compatible with those issues. Obviouslymonitoring has to be improved, especially for large projects (i.e. more than EUR 2 bn, as a rule ofthumb).Completion reports were produced for only five of the 29 projects evaluated, of which two projectswere considered good, two satisfactory and one poor.Two other projects were closed administratively, i.e. without a completion report. Questionnaires fortwo other projects were sent to the Promoters but not answered nor was this followed up by the Bank.22
Page 6 and 7: completion was delayed, expected tr
Page 8 and 9: Table of RecommendationsEV Recommen
Page 10 and 11: The Bank’s portfolio, as measured
Page 12 and 13: Rail Share of EU Transport Market 1
Page 14 and 15: The Maastricht Treaty and subsequen
Page 16 and 17: 3. Relevance/Effectiveness of Proje
Page 18 and 19: The main causes for the delays were
Page 20 and 21: evidence. What is more, even if suc
Page 22 and 23: maintain and operate their assets p
Page 26 and 27: In the final analysis, the Bank has
Page 28 and 29: ANNEX 1EVALUATION CRITERIAProject p
Page 30 and 31: EUROPEAN INVESTMENT BANKOPERATIONS

Evaluation of Railway Projects in the European Union

Create successful ePaper yourself

Delete template?

Save as template?