Evaluation of the Tuberculosis Control Assistance Program (TB CAP)

The PMU should adopt a definition of accrual that allows for estimated expenditures to be reportedunofficially to allow for better estimating of progress and future financial needs.Financial ReportingUSAID reports that day-to-day PMU forecasts of funding requirements in some cases seem unreliable; forexample, costly activities were known to have begun but forecasts for the costs are unrealistically low. Onthe other hand, independent auditors have confirmed that TB CAP is presenting its annual financialstatements to USAID fairly, and USAID perceives annual PMU financial reporting as being generallytimely. The PMU has been informeding USAID of possible delays in advance, reasonably explained whythe delay might occur, and receiving USAID approval to deliver later than expected if necessary.It is less clear if the detailed financial information presented in TB CAP annual reports is reliableassessing progress. Figures often change from year to year without explanation. The number andfrequency of changes in TB CAP annual reports suggests that the figures are not definite at the time ofreporting and therefore may not be accurate. Furthermore, these are often figures that would be expectednot to change (such as a country budget approved two years ago), which raises questions about thesoundness of PMU financial management practices. Among many examples, more than half the approvedAPA2 budgets for 15 countries listed in the year 2 report changed in the year 3 report withoutexplanation, and the APA1 Approved Management Budget changed in each annual progress report (from$1.14 million to $1.05 million to $1.13 million).Since the TB CAP project seems to take longer than anticipated to disburse approved budgets, annualspending is an important indicator for monitoring progress. Actual spending for TB CAP projects in agiven year, however, is not possible to determine. For countries, TB CAP progress reports calculatespending by comparing all country budgets (including budgets from previous years) with all countryexpenditures (including expenditures from previous years). As a result, the spending reported in eachannual report reflects not spending against budget for that year but the cumulative average. Notsurprisingly, then, the average appears to be improving every year (54% in APA1, 66% in APA2, and73% in APA3) as more and more budgets and completed expenditures are back-loaded into the formula.Although not ideal, a truer measure of the annual change in spending is to look only at countries in theirfirst year, when they have no previous budgets or expenditures to weight the average. If this is done,spending for new projects has not improved: 54% in APA1 (10 countries), 54% in APA2 (9 countries),and 8.2% in APA3 (2 countries). For core projects in their first year, the trend is in fact regressing: 71.5%in APA1, 66% in APA2 and 58% in APA3. This effect is no doubt heavily influenced by the start dates ofprojects (many of which start late in a reporting year) and the trend in core projects of programmingmultiyear efforts within a forced single-year framework.E. MONITORING AND EVALUATIONAn independent evaluation found that M&E of activities was an area of weakness in the predecessorproject, and with the addition of an M&E specialist in TB CAP, this area is perceived as having vastlyimproved. In particular, the reporting on technical progress in countries and the impact of TB CAPprojects appears to be well accounted for. The information (qualitative and, to a lesser extent,quantitative) generated by the PMU is valuable for communicating the value that TB CAP has forcountries and the international community. This reporting is considered to be timely and of high quality.M&E of project finances is also much better, although the PMU and partners must continue to improvethe speed and reliability of financial reporting and refine financial indicators to ensure they require only aminimum burden to generate.In other respects, however, little has been done to address recommendations related to M&E from thatearlier evaluation, particularly monitoring the responsibilities of the partners and evaluating project12 EVALUATION OF THE TUBERCULOSIS CONTROL ASSISTANCE PROGRAM (TB CAP)