A Model for Evaluation of Professional Organizations Using AHP ...

International Review of Business Research PapersVol. 2 No. 1. May 2006, Pp. 16 - 29A Model for Evaluation of Professional Organizations Using AHP withNon-equivalence of Experts’ Rating Confidence.Faith-Michael E. Uzoka * and Geoffrey G. Seleka * *Professional organizations play a critical role in the growth and development ofa nation. The task of effective management of these organizations depends on theeffective management of performance and results. A critical factor towards theachievement of business excellence is the development and implementation of amethodical process of performance measurement. Attempts have been made inthe past to measure performance based on quantitative financial measures, suchas sales turnover, profit, and return on investment, among others. Less emphasishas been placed on the qualitative components of performance measurement. Inthis study, a computational model is developed for the performance evaluation ofprofessional organizations using and enhancement of classical method ofAnalytic Hierarchy Process (AHP) to take cognizance of the confidence level ofthe experts rating the decision alternatives. The results of the study show thatwhile AHP can effectively combine quantitative and qualitative information toeffectively refine the results of an initial evaluation in order to achieve somedegree of optimality in the evaluation of professional organizations, the level ofconfidence of the experts provides a further refinement of the results in anattempt to normalize the human component of the AHP process.Field of Research: Management Information Systems1. IntroductionThe world order is increasingly tending towards specialization as nations and individuals tendto focus more on the areas of production and operation where they have relative advantage.This has led to an increase in the number of professional organizations world-wide. A keyattribute of these organizations is that they are mostly non-profit in nature. They aredistinguished from other organizations by the unique services they render to members, andthe society as a whole. Members of professional organizations posses some specializedknowledge that may be of more than ordinary complexity, which is usually acquired throughrigorous and formal study.*Dr. Faith-Michael Uzoka, Business Information Systems, Department of Accounting andFinance, University of Botswana, Email: uzokaf@mopipi.ub.bw** Mr Geoffrey .G. Seleka, Business Information Systems, Department of Accounting andFinance, University of Botswana, Email: selekagg@mopipi.ub.bw

Uzoka and Seleka 17They are guided by certain values and standards, which are usually closely spelt out in a codeof practice, or code of conduct, to which all members must commit themselves and alsoprovide a platform for interaction with government or other authorities in matters concerningthe profession (Kolade 1998). They provide training and certifying examination of studentsin the profession as well as the continuing professional update and development of members,and promote the advancement of the profession through research, publication of journals andperiodicals on matters that relate to the profession.The performance of any organization is largely dependent on the realization of itsdistinguishing objectives. In the past, some performance measurement systems have beendeveloped, but these systems have been highly structured and quantitative, or global innature, attempting to meet the needs of every industry. Such global systems lose the specificsrequired for given industries. Extreme concern for global optimization in decision supportsystems could be counter productive because of the forced translation of quantities which arenaturally expressed in different metrics into a common metric [Sands and Wiskoff 1996]. Itis noted in [Winston 1996] that no one model is able to effectively cater for the decisionneeds of two industries. Professional organizations are unique due to their non-profit natureand membership orientation. Revenue growth is minimal, while members’ expectations fortop quality professional services remain high. Balancing these realities has been, and willcontinue to be the most significant challenge facing the leadership of these organizations. It isobserved (Kanji 2001) that effective organizational management depends on the effectivemanagement of performance and results. In order to achieve business excellence, it ispertinent that an organization develops a system for performance measurement.This study attempts to develop a performance evaluation model for professional organizationsusing the AHP methodology. The AHP uses prioritization of variables by domain expertswhich is used rank decision alternatives based on ordinal or nominal scales. However, theevaluation does not take cognizance of the experiential confidence levels of the experts whorate the organizations. We propose an AHP methodology that integrates the overallconfidence level of the expert in his rating of organizations. In Section 2.0, we review someliterature relating to performance/productivity measurement. Section 3.0 addresses themethodology of our study, while Section 4.0 presents the results and discussion of findings.In Section 5.0, we present some conclusions and suggestions for further studies.2. Literature ReviewProductivity level is of critical importance to enterprise competitiveness. Many approacheshave been adopted for the measurement of organizational productivity. The method ofmeasurement is often determined by the purpose of the productivity analysis, which mayinclude: comparing an enterprise with sector leaders or competitors, determining the relativeperformance of departments and workers, comparing relative benefits of various types of17

Uzoka and Seleka 18input for collective bargaining, and gain sharing. One method of performance measurementis the use of inter-firm comparison (IFC) and benchmarking (NEDO 1981, ILO 1996). IFCinvolves the voluntary inter-firm exchange of information regarding costs, performance,efficiency and other relevant data for purposes of comparative analysis, which showsmanagement how its firm’s performance compares with others in the same industry. IFC aimsat drawing management’s attention to areas of comparative weakness and strength within thebusiness, and to give management an objective basis for judging progress and effectiveness.Benchmarking on the other hand, goes beyond IFC. It is a collaborative approach wherebyseveral organizations share experience and help each other in productivity assessment byidentifying specific activities and practices that can, and ought to be changed in order toinitiate performance improvements that are sustainable.Kurosawa (1980) proposed a productivity measurement model, which focuses on thestructure of the enterprise, emphasizing an information system that measures productivityaccording to the decision making hierarchy of the organization. Productivity of an individualor unit is seen as the value added to the organization by the individual or unit. Lawlor (1985)defined productivity measurement in terms of how organizations satisfy the followingparameters: objectives, efficiency, effectiveness, comparability and progressive trends.Bernolak (1993) proposed the combination of typical productivity and profitability ratios inestablishing the overall performance of the organization. The integrated analysis ofprofitability and productivity makes the application of productivity theory very practical forenterprise managers. While profitability measures indicate the combined short-term effects ofprice factors, the long-run success of enterprises is determined by proper resource utilization,which is the productivity factor.The logic model is proposed in (Hatry et al. 1996, McLaughlin and Jordan 1999, FairfaxCounty 2003) as a means of developing performance measures for organizations. The logicmodel captures a series of “If-then” changes that the programme intends to influence throughits inputs, activities and outputs. It is usually derived from a matrix of inputs, outputs andoutcomes, as depicted in Table 1. Logic models permit the use of eclectic approach toperformance measurement, which permits either a walk forward from inputs to outcomes, ora walk backwards from outcomes to inputs.Productivity improvement is the result of the combined efforts of many different stakeholderswith different objectives and perceptions about organizational effectiveness. This means thatthe results of measurement of different parameters should be fed back to all participants tostimulate improvement. ILO (1996) recommended that a “family of measures” should beused since no part of an organization is so simple that it can be measured adequately by oneindicator. By considering only a single measure or a few measures concentrated in onedimension, almost any group can be successful in the short term by optimising the selectedmeasure at the expense of other important measures. In (Fairfax County 2003), the family ofmeasures approach was used in order to present a balanced picture of performance byutilizing a broad spectrum of indicators, which include input, output, efficiency, servicequality, outcome, among others.18

Uzoka and Seleka 19Table 1– A Logic Model MatrixINPUTS OUTPUTS OUTCOMESWhat weinvestStaffDollarsVolunteersMaterialsEquipmentWhat we do(Activities/Strategies)WorkshopsOutreachInspectionsAssessmentsShort-termAwarenessKnowledgeAttitudesSkillsMediumtermBehaviourDecisionsPoliciesLong-termConditionsSafetyEconomicSocialEnvironmeCertain difficulties arise in organizational performance measurement, which border on themethod of measurement and parameters considered. Such problems (ILO 1996) include:how to combine different types of input into one acceptable denominator; how to embraceall resources and business activities; how to deal with qualitative changes in input or outputover time; how to interpret measurement results; how to indicate accountability and provideclear signals for managerial decisions and action; and how to make productivity measurementcredible and easily understandable to everybody concerned in various parts of theorganization.The use of the logic model and family of measures has been cumbersome. This is because theprocess is usually long, and could become combinatorially explosive when the evaluationvariables are numerous. Productivity measurement and other multi-criteria evaluationdecisions are hierarchically structured (Liu and Wei 2000, Tsoi 2001). The AnalyticHierarchy Process, proposed by Saaty (1981), uses mathematical algorithms to transformqualitative subjective judgments into quantitative data, which produces a computationalmodel that serves as input into an evaluation process. Other methods of multi-criteriadecision analysis (MCDA) include: Preference Ranking Organization METHod for EnrichmentEvaluations [PROMETHEE] (Brans and Vincke, 1985), Technique for Order Preference bySimilarity to Ideal Solution [TOPSIS] (Hwang and Yoon, 1981), Ordered WeightedAveraging [OWA] (Yager, 1988), the Simple Additive Weighting [SAW] (Hwang andYoon, 1981), Data Envelopment Analysis [DEA] (Charnes et al. 1978), the ELECTRE(Roux and Elloy, 1985), and Fuzzy Ranking [FR] method (Critto et al. 2002). AHP ispreferred to other MCDA methods because it meets the MCDA method properties ofinteraction, weighting, dominance, and scaling (Naumann, 1998).With the AHP the analyststructures a problem hierarchically and then, through an associated measurement-anddecompositionprocess, determines the relative priorities consistent with overall objectives(Hartwich and Janssen 2000). However AHP utilizes domain experts’ judgments in settingcriteria priorities and rating decision alternatives. There are differences in the professional andexperiential knowledge of the domain experts who carry out the ratings. This study is acontribution to recent attempts to recognize these differences in prioritizing decision variablesand rating of the decision alternatives ( Mikhailov, 2004; Beynon, 2005).19

Uzoka and Seleka 203. MethodologyThe study population consists of professional organizations in Nigeria. As at the time of thestudy, there were thirty seven professional bodies in Nigeria, out of which only nineteen wereregistered with the Association of Professional Bodies of Nigeria (APBN). Ten out of thesewere randomly sampled for the purpose of the research. A total of one hundredquestionnaires were administered to professionals in these organizations, with various levels ofexperience and qualifications (Table 2) who made judgments regarding relative levels ofimportance of the evaluation decision variables. The utilization of person-to-person contactensured proper filling and return of seventy two of the questionnaires. The followingparameters were considered in determining the performance of the professional bodies:Professional Development, Finance and Budget, Membership Services, and Human ResourcesManagementTable 2 – Distribution of RespondentsAge (x years) x < 20 20 ≤ x < 40 40 ≤ x < 60 x ≥ 603% 34% 49% 14%Experience (y years) y < 5 5 ≤ y < 15 15 ≤ y < 25 y ≥ 2516% 52% 27% 5%Highest Qualification < B.Sc. B.Sc. (or equivalent) M.Sc. (or equivalent) PhD6% 54% 37% 3%The decision support base for determining performance of professional organization is built onthe AHP principle (Saaty 1981, Saaty and Vargas 1998). The Analytic Hierarchy Process(AHP) attempts to handle multi-criteria decisions in order to determine the relativeimportance of the evaluation variables in the decision process. When the amount of variablesto be combined in a decision process becomes many, the expert’s preference may decay andmay not be able to make effective comparison. This is because the human mind caneffectively handle at most 7 ± 2 pieces of information at the same time, and becomesinefficient as the number of information increases (Miller 1956). The AHP deals withdependence among variables or clusters of a decision structure to combine structured andunstructured information. The AHP is based on three principles: decomposition,measurement of preferences and synthesis.Decomposition breaks down a problem into manageable elements that are treatedindividually. It begins with implicit description of the problem (the goal) and proceedslogically to the criteria (or states of nature), in terms of which outcomes are evaluated. Theresult of this phase is a hierarchical structure, consisting of levels for grouping issues togetheras to their importance or influence with repect to the elements in the adjacent levels above.The decomposition of the professional organization evaluation decision into hierarchy ispresented in Figure 1, while the hierarchical structure is shown in Figure 2.20

Uzoka and Seleka 21Level 1 –Goal Level 2-criteria Level 3 – VariablesPerformancemeasurementof ProfessionalProfessionalDevelopment (PFD)Finance and Budget(FNB)Codes and standards (CDS), position papers(PSP), Technical visits (TCV), Workshops andseminars (WSS), conferences (CFS), Researchand Publications (RPB), Mandatory ContinuingProfessional Development (MCP)Donations/Benevolence (DBV), ProfitableInvestments (PFI), Budget Performance (BGP),Revenue Generation (RGT)Organization Membership Jobs/IT placement (JIT), ProfessionalServices (MBS) Examination (PFE), Arbitrations (ARB),(PMPO)Membership Drive (MBD), Subscriptions (SBS).Human Resources Staff Recruitment (STR), Staff Training andManagement Development (STD), Staff Compensation(STC), Staff Motivation (STM), StaffPerformance Evaluation (SPE)Figure 1 – Hierarchy of Performance Measurement CriteriaMeasurement of preferences involves a pairwise comparison of decision variables, which areverbal statements about the strength of importance of a variable over another, representednumerically on an absolute scale. The comparison is done from the top level of the hierarchyto the bottom level in order to establish the overall priority index. If two varaiables are ofequal importance, the rating of the comparison is 1. If variable A is strongly more importantthan variable B, then the rating of the comparison (AB) could be 7. If A is weakly moreimportant, the rating is 3. The reciprocal of the ratings show the converse of the relativeimportance. To obtain a fair judgment of the comparisons, mean values of the comparisonratings made by the seventy two professionals were used in formulating the pairwisecomparison (PWC) matrix of the variables in levels 2 and 3 of Figure 1. The Pairwisecomparison (PWC) matrix for the level 2 criteria is shown in Table 3, while that of level 3variables (for professional development) is shown in Table 4.21

Uzoka and Seleka 22Performance DecisionLevel 2ProfessionalDevelopment Finance and Budget Membership Services Human Resources ManagementDBVPFIBGPRGTSTR STD STC STM SPECDS PSD TCV WSS CFS RPB MCPLevel 3JIT PFE ARB MBD SBSFigure 2 – Hierarchical structure of evaluation variablesTable 3 – Level 2 PWC MatrixPFD FNB MBS HRMPFD 1 6 5 7FNB 1/6 1 ½ 3MBS 1/5 2 1 4HRM 1/7 1/3 ¼ 1Table 4 – PWC Matrix for ProfessionalDevelopment (Level 3)CDS PSP TCV WSS CFS RPB MCPCDS 1 5 9 ¼ 7 5 ¼PSP 1/5 1 8 1/3 3 2 ¼TCV 1/9 1/8 1 1/7 1/8 1/6 1/8WSS 4 3 7 1 3 3 ¼CFS 1/7 1/3 8 1/3 1 1/3 1/6RPB 1/5 ½ 6 1/3 3 1 1/6MCP 4 4 8 4 6 6 1Synthesis involves the computation of eigenvalues and the eigenvector. In synthesizing thejudgments, the values in each column of the PWC matrix are first added. Then each entry ineach column is divided by the total of the column to obtain the normalized matrix, whichpermits meaningful comparison among elements. Finally, adding the values in each row of thenormalized matrix and dividing the rows by the number of row entries gives the average over22

Uzoka and Seleka 23the rows. This synthesis yields the percentage of relative priorities, which is expressed in alinear form to give the eigenvector. The implication of the eigenvector is that it expresses therelative importance of an attribute over another in the minds of the decision maker. Theeigenvalues/ eigenvector computations can be expressed mathematically as follows:VEij=Tij. j(1)where E ij is the eigenvalue of cell {aij}, Vij is the value of PWC matrix for cell {aij}, andis the sum of the values on column jT.jThe eigenvector for variable k is a column vector given as:n∑Ekjj = 1λk=(2)nwhere k is the eigenvector value corresponding to variable k, Ekj is the eigenvalue of cell {akj},(j = 1,2, …,n), and n is the number of evaluation variablesThis initial table of eigenvalues and eigenvector helps to establish priority model. It isimportant to note that the pairwise comparisons are also carried out for elements of the subcriteria(variables) of all evaluation criteria (factors). The eigenvalues and eigenvector for thelevel 2 criteria are shown in Table 5.Table 5 – Level 2 Eigenvalues and EigenvectorPFD FNB MBS HRM EigenvectorPFD 0.662 0.643 0.741 0.467 0.6283FNB 0.111 0.107 0.074 0.200 0.1230MBS 0.132 0.214 0.148 0.267 0.1903HRM 0.095 0.036 0.037 0.067 0.0588The next step is the rating of the professional organizations based on the respective variables.The evaluation of the ten professional bodies used in this study was carried out using anevaluation sheet that was designed to generate evaluations in terms of the twenty onevariables identified in Figure 1. These evaluations were carried out by the seventy twoexperts. A crucial component of the questionnaire was the determination of the level ofconfidence x (0

Uzoka and Seleka 24n∑W i= λ R µj = 1jijij(3)where Wi is the weighted rating of organization i, j is the eigenvector value of variable j, Rij isthe rating of organization i on variable j, ij is the mean confidence level of the rating oforganization i on variable j.This forms the basis of ranking that leads to the eventual rating of the performance of theorganization.4. Results and AnalysisThe level 2 criteria evaluation gives an eigenvector, 1 that forms the Professional BodyEvaluation Factor Index (PBEFI) in respect of the level 2 criteria.PBEFI = 0.6283PFD + 0.1230FNB + 0.1903MBS + 0.0588HRM (4)Expanding (4) using the eigenvalues/vector matrices for the four factors, we have:PBEFI = 0.6283 (0.1991CDS + 0.0960PSP + 0.0193TCV + 0.1926WSS +0.0570CFS + 0.0714RPB + 0.3646MCP) + 0.1230 (0.0653DBV + 0.2385PFI +0.1128BGP + 0.6100RGT) + 0.1903 (0.0306JIT + 0.3382PFE + 0.0610ARB +0.3754MBD + 0.1948SBS) + 0.588 (0.1924STR + 0.3626STD + 0.0702STC +0.0936STM + 0.02394SPE) (5)This translates to:PBEFI = 0.1251CDS + 0.0603PSP + 0.0121TCV + 0.1210WSS + 0.0358CFS +0.0449RPB + 0.2291MCP + 0.0081DBV + 0.0293PFI + 0.0139BGP + 0.0750RGT+ 0.0058JIT + 0.0644PFE + 0.0116ARB + 0.0714MBD + 0.0371SBS +0.0113STR + 0.0213STD + 0.0041STC + 0.0055STM + 0.0014SPE (6)The initial evaluation of the ten professional organizations before the application of the AHPis presented in Table 6. These evaluations were carried out by the seventy two experts.Evaluation bias was localized by asking each professional to evaluate his organization based onthe twenty variables identified in Figure 1. The evaluations were carried out based on a fivepoint qualitative likert scale as follows: poor-1, fair-2, good-3, very good-4, excellent-5. Asimple average of the evaluations was taken in each case, and approximated to the nearestnatural number on the likert scale. For purposes of anonymity, the organizations arerepresented by hypothetical organization identification numbers. After the application ofPBEFI generated by AHP, the final evaluation was arrived at as shown in Table 7.Table 6 – Initial Evaluation of OrganizationsOrg CDS PSP TCV WSS CFS RPB MCP DBV PFI BGP RGT JIT PFE ARB MBD SBS STR STD STC00001 3 4 5 3 5 3 2 4 4 2 3 4 4 3 4 3 2 2 300002 4 3 4 5 5 4 4 3 4 3 4 3 5 2 4 4 4 3 400003 3 2 3 3 4 2 2 2 4 4 3 2 3 2 3 3 3 3 300004 3 1 3 4 5 3 1 3 3 2 3 2 3 2 4 3 3 3 400005 2 2 3 2 3 2 1 1 3 2 2 1 3 1 3 2 2 2 200006 2 1 3 3 3 2 1 2 2 2 2 2 3 2 3 2 3 2 200007 3 3 4 2 4 3 1 2 3 2242 2 3 2 3 2 2 2 200008 1 2 3 3 3 2 1 1 3 2 2 1 4 1 3 2 3 2 200009 1 2 2 2 3 1 1 2 2 2 1 1 3 1 2 1 2 2 200010 2 2 3 2 2 1 1 1 2 2 2 2 3 2 3 2 2 2 2

Uzoka and Seleka 25The ratings in Table 6 show that the organization with identification number 0002 rankedfirst, while 00001 ranked second, and so on. After the application of the PBEFI on Table 6,the final ranking shows that organizations 00003 and 00004, which were both tied on thethird position in the initial ranking, now rank third and fourth respectively (Table 7).Organization with identification number 00006, which ranked fourth in the initial ranking,now ranks seventh. The results show that AHP is able to provide a mathematical formalismfor refining the results of the initial rakings of the professional organization, which could serveas an initial basis for decision making concerning the performance of these professionalorganizations.With the introduction of the confidence levels of the experts on the ratings of theorganizations (Table 8), a further refinement of the evaluation results is carried out throughthe scaling of the results obtained in Table 7 using the relationship:1S = A. (7)TCwhere S is the final score matrix, A is the matrix shown in Table 7 less the last two columns,while C T is the transpose of the matrix shown in Table 8.Table 7 – Evaluation Result based on AHPOrg. CDS PSP TCV WSS CFS RPB MCP DBV PFI BGP RGT JIT PFE ARB MBD SBS STR STD STC STM00001 0.375 0.241 0.061 0.363 0.179 0.135 0.458 0.032 0.117 0.028 0.225 0.023 0.258 0.035 0.286 0.111 0.023 0.043 0.012 0.000002 0.500 0.181 0.048 0.605 0.179 0.180 0.916 0.024 0.117 0.042 0.300 0.017 0.322 0.023 0.286 0.148 0.045 0.064 0.016 0.000003 0.375 0.121 0.036 0.363 0.143 0.088 0.458 0.016 0.117 0.056 0.225 0.012 0.193 0.023 0.214 0.111 0.034 0.064 0.012 0.000004 0.375 0.060 0.036 0.484 0.179 0.135 0.229 0.024 0.088 0.028 0.225 0.012 0.193 0.023 0.286 0.111 0.034 0.064 0.016 0.000005 0.250 0.121 0.036 0.242 0.107 0.090 0.229 0.008 0.088 0.028 0.150 0.006 0.193 0.012 0.214 0.074 0.023 0.043 0.008 0.000006 0.250 0.060 0.036 0.363 0.107 0.090 0.229 0.016 0.059 0.028 0.150 0.012 0.193 0.023 0.214 0.074 0.034 0.043 0.008 0.000007 0.375 0.181 0.048 0.242 0.143 0.135 0.229 0.016 0.088 0.028 0.150 0.012 0.193 0.023 0.214 0.074 0.023 0.043 0.008 0.100008 0.125 0.121 0.036 0.363 0.107 0.090 0.229 0.008 0.088 0.028 0.500 0.006 0.258 0.012 0.214 0.074 0.034 0.043 0.008 0.000009 0.125 0.121 0.024 0.242 0.107 0.045 0.229 0.016 0.059 0.028 0.075 0.006 0.193 0.012 0.143 0.037 0.023 0.043 0.008 0.000010 0.250 0.121 0.036 0.242 0.072 0.045 0.229 0.008 0.059 0.028 0.150 0.012 0.193 0.023 0.214 0.074 0.023 0.043 0.008 0.0The implication of the relationship in (7) is that if the average confidence level of the raterson a given variable is high, then their ratings would have averaged close to the realistic value,but if the average confidence level is low, then the reciprocal of the confidence value wouldeither increase or reduce the ratings accordingly in order to approximate the realistic value.Table 8 – Average Confidence Levels of Experts on the RatingsOrg CDS PSP TCV WSS CFS RPB MCP DBV PFI BGP RGT JIT PFE ARB MBD SBS STR STD STC STM00001 0.9 0.7 0.8 0.7 0.9 0.6 0.6 0.7 0.7 0.8 0.7 0.5 0.5 0.6 0.6 0.9 0.4 0.5 0.6 0.600002 0.7 0.6 0.7 0.9 0.8 0.7 0.7 0.8 0.5 0.4 0.6 0.9 0.7 0.5 0.8 0.8 0.7 0.6 0.6 0.400003 0.5 0.5 0.6 0.6 0.7 0.5 0.5 0.6 0.7 0.5 0.6 0.8 0.8 0.7 0.7 0.7 0.6 0.8 0.9 0.700004 0.6 0.8 0.8 0.7 0.8 0.5 0.3 0.6 0.8 0.6 0.6 0.6 0.6 0.7 0.5 0.8 0.5 0.8 0.5 0.400005 0.4 0.7 0.7 0.7 0.5 0.4 0.4 0.7 0.5 0.6 0.6 0.5 0.6 0.9 0.8 0.5 0.8 0.5 0.4 0.800006 0.9 0.6 0.6 0.8 0.5 0.5 0.7 0.4 0.7 0.7 0.6 0.4 0.7 0.8 0.6 0.6 0.9 0.7 0.6 0.600007 0.8 0.6 0.7 0.7 0.6 0.6 0.5 0.6 0.8 0.8 0.4 0.6 0.5 0.5 0.9 0.8 0.7 0.7 0.7 0.900008 0.6 0.8 0.6 0.8 0.5 0.3 0.8 0.6 0.9 0.4 0.7 0.6 0.9 0.7 0.4 0.6 0.4 0.8 0.5 0.700009 0.7 0.5 0.6 0.8 0.6 0.4 0.6 0.9 0.6 0.5 25 0.8 0.8 0.8 0.7 0.6 0.9 0.7 0.6 0.8 0.800010 0.9 0.6 0.7 0.7 0.7 0.5 0.4 0.6 0.4 0.5 0.7 0.6 0.6 0.8 0.7 0.8 0.9 0.9 0.6 0.6

Uzoka and Seleka 26Applying equation (7) to Tables 7 and 8 produces the final evaluation result, shown in Table9. This is a refinement of the evaluation result obtained by AHP to include the confidencelevels of the experts evaluating the organizations.Table 9 – Final Evaluation ResultOrg Id Final Rating Rank00001 4.518893 400002 5.732718 100003 4.574631 300004 4.577952 200005 3.607595 700006 2.979794 800007 3.664917 600008 3.713016 500009 2.380044 1000010 2.942111 9From Table 9, it could be observed that the organizations with ids. 00002, 00003, 00009,and 00010 still retained their first, third, tenth and ninth positions respectively. However,the following organization pairs interchanged positions respectively: {00001, 00004},{00005, 00006}, {00007, 00008}. The implication is that the initial ratings producedresults which may have been affected by the degree of confidence of the experts who ratedthe organizations. However, this was refined by using the confidence levels to deflate theinitial ratings.The study further reveals that high premium is placed on professional development, which hasan eigenvector value of 62.83% (equation 4). In particular, professional organizationsemphasize codes and standards (12.51%), and workshops/seminars (12.10%) as keydeterminants of performance.5. Conclusion and RecommendationsMeasuring performance of a service oriented organization such as a professional body is acomplex managerial function that requires the combination of quantitative and qualitativedecision variables. These variables are in most cases, hierarchically structured. The decisionsupport model proposed in this study, constructs a hierarchy of decision criteria, which aremainly qualitative in nature. This can be effectively used to carry out the performancemeasurement of professional organizations and a comparative measurement, with a view toranking the professional organizations. Furthermore, the classical method of AnalyticHierarchy Process is used to construct a pairwise comparison matrix of all the decisionvariables to obtain a matrix of eigenvalues and eigenvector, which aids the eventual ranking ofthe professional organizations. This was further refined based on the levels of confidence ofthe experts rating the organizations. The level of confidence is affected by the relevance ofthe expert’s discipline to the organization in question, his years of experience, qualifications,and other organizational factors. The results of the study show that a stepwise refinement ofthe results of the initial evaluation process can be achieved to attain some level of optimality26

Uzoka and Seleka 27in the decision process. The functionality of the model has been demonstrated in order toshow its ability to assist in carrying out comparative assessment of professional organizations.The study reported in this paper has considered the measurement of performance ofprofessional organizations. The model can be expanded in future for the purpose ofgeneralization to include the evaluation of other sectors of the nation’s economy such asmanufacturing organizations and government establishments. Furthermore, future study couldconsider the use of Factor Analysis in the factorizing the decision variables and determiningtheir relative contributions to a factor model. An integration of the levels of confidence of thedecision experts in determining the pairwise comparison of the variables is also a subject offurther study.27

Uzoka and Seleka 28ReferencesBernolak, I. 1993. “Effective measurement and successful elements of company productivity:The basis of competitiveness and world prosperity”, Eight World Productivity Congress,Stockholm, pp. 4-6.Beynon, M.J. 2005. “A method of Aggregation in DS/AHP for Group Decision Making withthe Non-equivalent importance of Individuals in the Group”, Computers and OperationsResearch, Vol. 32, pp. 1881-1896Brans, J.P. and Vincke, P. 1985. “A Preference Ranking Organization Method: ThePROMETHEE Method for MCDM”, Management Science. Vol. 31, No.6, pp. 647-656Charnes, A., Cooper, W. and Rhodes, E. 1978. “Measuring the Efficiency of DecisionMaking Units” European Journal of Operations Research. Vol. 2, pp. 429-444Critto, A, Nadal, N, Samiolo, M, Carlon, C, Silvoni, S, Foramti, S, and Marcomini, A.2002. “DESYRE- Decision Support sYstem for REHabilitation of contaminated sites:objectives and structure” Proceedings of the Integrated Assessment and Decision SupportConference, Lugano, Switzerland.Fairfax County. 2003. “A manual for performance measurement, Department ofManagement and Budget” seventh edition,http://www.co.fairfax.va.us/gov/omb/perf_measure.htmHartwich, F. and Janssen, W. 2000. “Setting Research Priorities: An Example fromAgriculture using the Analytic Hierarchy Process”, Journal of Research Evaluation, Band Dez,Heft (http://www.uni-hohenheim.de)Hatry H., Van Houten, T., Plantz M.C., and Greenway M.T. 1996. “Measuring ProgramOutcomes: A Practical Approach”, United States of America, Alexandria.Hwang, C-L. and Yoon, K. 1981. “Multiple Attribute Decision Making. Methods andApplications. A State-of-the-Art Survey”. Springer-Verlag Lecture Notes, No. 186ILO. 1996. “Management Consulting: A guide to the profession”, (third edition) Geneva,International Labour Office.Kanji, G. K. 2001. “Kanji Certification of Performance Measurement (KCPM)”,http.//www.gopal_kanji.com/performance/index.htmKolade, C. O. 1998. “Working for Business Integrity in Nigeria: Roles of the Professional,Association of Professional Bodies of Nigeria (APBN)”, Fourth Annual Public Lecture, Lagos,Kurosawa, K. 1980. “Structural approach to the concept and measurement of productivity(2)”, Studies in Economics Science/ Keizai Shushi, Tokyo, Nihon University, Vol. 50, No. 2,pp. 96-135.28

Uzoka and Seleka 29Lawlor, A. 1985. “Productivity improvement manual”, Aldershot, Hampshire; Gower.Liu, P-H. and Wei, C-C. 2000. “A Group Decision Making Technique for Appraising thePerformance of Organizations”, International Journal of the Computer, the Internet andManagement, Vol. 8, No 2, pp. 39 – 49.McLaughlin, J. A. and Jordan, G. B. 1999. “Logic models: A tool for telling yourprogramme’s performance story”, Evaluation and Programme Planning, pp 65-72.Mikhailov, L. 2004. “Group Prioritization in the AHP by Fuzzy Preference ProgrammingMethod”, Computers and Operations Research, Vol. 31, pp 293- 301.Miller, G.A. 1956. “The Magical Number even, Plus or Minus Two: Some Limits on OurCapacity for Processing Information”, Psychological Review, Vol. 63, pp. 81-97Naumann, F. 1998. “Data Fusion and Data Quality”, Proceedings of the New Techniquesand Technologies for Statistics Seminar, Sorrento, Italy: 147-154NEDO. 1981.”Tool making: A comparison of UK and West Germany Companies”, London,National Economic Development Office.Roux, O. and Elloy, J.P. 1985. “ELECTRE: A Language using Control Structure Expressionsto Specify Synchronization”, Proceedings of the 1985 ACM Annual Conference on theRange of Computing: Mid-80’s Perspective, Colorado, USA: pp.240-245.Saaty, T.L. 1981. “Priorities in Systems with Feedback”, International Journal of Systems,Measurements and Decisions, Vol. 1, pp. 24-38Saaty, T.L. and Vargas, L.G. 1998, “Diagnosis with Dependent Symptoms: Bayes Theoremand the Analytic Hierarchy Process”, Operations Research, Vol. 46, Iss. 4, pp. 491 – 503Sands, W.A. and Wiskoff, M.F. 1996. “Expert System Planning; German Federal ArmedForces Psychological Service”, 38 th Annual Conference of the International Military TestingAssociation, U.S.ATsoi, R.H. 2001, “Using Analytical Hierarchy Process (AHP) Method to Prioritize HumanResources in Substitution Problem”, International Journal of the Computer, the Internet andManagement; Vol. 9, No. 1, pp. 36 – 45.Winston, B. 1996. “The Development of a General Measure of Performance”, 38 th AnnualConference of the International Military Testing Association; U.S.AYager, R.R. 1988. “On Ordered Weighted Averaging Aggregation Operators in MulticriteriaDecision Making”, IEEE Transactions on Systems Management and Cybernetics, Vol. 18, pp.183-190.29