Compendium of Country Examples and Lessons Learned from ...

“Considerations for assessment” designed to provide additional guidance for the assessor on thecritical aspect to be examined.The findings should be recorded in a short narrative report, which discusses the probable causes fordata that do not show the expected degree of compliance.As noted in the User‟s Guide, the use of the CPIs will need to be determined on a country basis takinginto consideration the specific capacities and issues that exist in the country and the decisions of thoseparticipating in the assessment including timing, resources needed, costs involved for collecting andanalysing data and conducting interviews and surveys. However, the use of the CPIs is considered tobe a critical aspect of monitoring the effectiveness of reforms that may be introduced to addressweaknesses in the overall system.Country experiencesWhen reflecting the experiences of the pilot countries on the assessment of the CPIs it was reportedthat the exercise has helped countries to identify performance issues and determine gaps in theirsystems. Keeping ownership in the implementation of the Methodology is essential for achieving theoutcome and for future commitments. This is best assured by self-assessments and joint assessments.However, it was suggested at the JV meeting in Copenhagen that further guidance, simplification ofthe CPIs and better alignment of the procurement assessment framework with the PEFA/PFMframework is needed. Consequently, at this very early stage of applying the compliance/performanceindicator system, the following sections will focus on presenting useful background information andon sharing experiences and challenges faced.The sampling phaseBackground informationIn order to obtain a valid picture of the performance of the procurement system, one of the mostimportant steps is to carefully consider the sample of cases and procuring entities (PEs) to be assessed.The sample needs to provide enough information in order to be able to draw conclusions, which can beregarded as valid at an aggregate level and be generalised to the procurement system as a whole.Different ways of sampling 16Should the distribution of cases be known (percentage of goods/services/works; method ofprocurement; contract amount; procurement at national and sub-national level, etc.) the method of“representative sampling” can be used by analysing a sufficient number of cases within eachcategory.It is also possible to draw a “random sample” which will need to be of sufficient size, so as to ensurethat the data collected can be seen as representative. Random sampling is in many ways the mostadequate type of sampling, given that it is possible to draw a large enough sample and that thepopulation it is drawn from is known and defined.16To assist in carrying out the pilot assessments, the OECD Secretariat provided the OECD GuidanceNote on Data Collection and Verification.COMPENDIUM OF COUNTRY EXAMPLES AND LESSONS LEARNED […] - OECD 2008 31