20 Sandy TautP5 Complete and Fair Assessment <strong>The</strong> evaluation should be complete and fair <strong>in</strong> itsexam<strong>in</strong>ation and record<strong>in</strong>g of strengths and weaknesses of the program be<strong>in</strong>gevaluated, so that strengths can be built upon and problem areas addressed.Standard P5, Complete and Fair Assessment, exam<strong>in</strong><strong>in</strong>g a program to f<strong>in</strong>d both strengthsand weaknesses, seems a goal for evaluators everywhere, despite multiple factors that might get<strong>in</strong> the way of this Standard, for example, pressures exerted by powerful stakeholders to stresseither strengths or weaknesses. <strong>The</strong>se pressures could be said to have more impact on evaluators<strong>in</strong> hierarchical and collectivist societies (also see analysis of Standard P7,‘Conflict of Interest’).P6 Disclosure of F<strong>in</strong>d<strong>in</strong>gs <strong>The</strong> formal parties to an evaluation should ensure thatthe full set of evaluation f<strong>in</strong>d<strong>in</strong>gs along with pert<strong>in</strong>ent limitations are made accessibleto the persons affected by the evaluation, and any others with expressed legal rights toreceive the results.Standard P6, Disclosure of F<strong>in</strong>d<strong>in</strong>gs, displays the right to know which is legally grantedto United States citizens as part of the Bill of Rights. As an example of how values provideultimate standards, Peoples & Bailey (1997, p. 28) mention “the American emphasis on certa<strong>in</strong>rights of <strong>in</strong>dividuals as embodied <strong>in</strong> the Bill of Rights <strong>in</strong> the Constitution […] Freedom of thepress, of speech, and of religion are ultimate standards […].” In cultures other than the UnitedStates, these issues do not play such a vital role <strong>in</strong> an <strong>in</strong>dividual’s rights (see Beywl, 1996). Forexample, Hendricks & Conner (1995, p. 85) po<strong>in</strong>t out that Australia has a Freedom ofInformation Act but “it would be difficult (even under this act) to allow all relevant stakeholdersto have access to evaluative <strong>in</strong>formation, and …actively dissem<strong>in</strong>ate that <strong>in</strong>formation tostakeholders.” In addition, the “code of directness, openness, and completeness” (Jo<strong>in</strong>tCommittee, 1994, p. 109) might not be <strong>in</strong> accordance with the preference (and superiority) of<strong>in</strong>direct communication and the existence of hierarchical structures <strong>in</strong> certa<strong>in</strong> cultures. Jang(2000, p. 15) states that <strong>in</strong> South Korea, “the task of ‘fully’ <strong>in</strong>form<strong>in</strong>g research subjects who areunfamiliar with the purposes and procedures of the study is not the first concern.” Likewise,absence of such disclosure might not, as claimed <strong>in</strong> the <strong>Standards</strong> (p. 109), threaten theevaluation’s credibility. To the contrary, the absence of disclosure may preserve the evaluation'scredibility <strong>in</strong> some cultures. Smith et al. (1993, p. 10) offer an example of the acceptable absenceof disclosure when they refer to both Maltese and Indian cultures. For these cultures, thepossibility of “plac<strong>in</strong>g someone <strong>in</strong> jeopardy” (e.g., by negative evaluation f<strong>in</strong>d<strong>in</strong>gs) is nevercommunicated <strong>in</strong> writ<strong>in</strong>g, thus limit<strong>in</strong>g the disclosure of f<strong>in</strong>d<strong>in</strong>gs.
Cross-Cultural Transferability of the <strong>Program</strong> <strong>Evaluation</strong> <strong>Standards</strong> 21P7 Conflict of InterestConflict of <strong>in</strong>terest should be dealt with openlyand honestly, so that it does not compromise the evaluation processes and results.Standard P7, Conflict of Interest, is another Standard reflect<strong>in</strong>g North American values.<strong>The</strong>re are different cultural standards for deal<strong>in</strong>g with conflicts. Group-orientation <strong>in</strong> collectivistcultures results <strong>in</strong> conflict avoidance or non-adversarial conflict resolution with<strong>in</strong> groups (seeBerry et al., 1997, Vol. 3, chapter 1, p. 26). In those cultures, it might be counterproductive toopenly address an exist<strong>in</strong>g conflict. In some cultures, direct communication is generally avoided(see Jang, 2000, p. 18). Another cross-culturally relevant problem with this Standard is thehigher <strong>in</strong>terdependency of people <strong>in</strong> collectivist societies. Personal friendships or professionalrelationships play a much more crucial role <strong>in</strong> these cultures than <strong>in</strong> <strong>in</strong>dividualistic societies suchas the U.S. As Jang (2000, p. 12) describes South Korean culture, “professionalism is def<strong>in</strong>edma<strong>in</strong>ly by an <strong>in</strong>dividual’s loyalty toward the group he or she serves.” In addition to the priorityof group <strong>in</strong>terest, Jang mentions the great impact of authority figures on op<strong>in</strong>ion formation (p.22). Accord<strong>in</strong>g to Smith et al. (1993, p. 10), Indian evaluators are “specifically chosen for apo<strong>in</strong>t of view consistent with the official position” and this would not be considered a conflict of<strong>in</strong>terest situation there. Standard P7 can therefore be considered unsuitable <strong>in</strong> certa<strong>in</strong> culturalcontexts.P8 Fiscal Responsibility <strong>The</strong> evaluator’s allocation and expenditure ofresources should reflect sound accountability procedures and otherwise be prudentand ethically responsible, so that expenditures are accounted for and appropriate.F<strong>in</strong>ally, Standard P8, Fiscal Responsibility, is seen as cross-culturally applicable.Accuracy <strong>Standards</strong>Across cultures, different degrees of importance are allocated to accuracy as part of anevaluation (see Nevo, 1984, p. 385 on the high priority of accuracy <strong>in</strong> Israeli evaluations). Databasedjudgements might be less credible than personal op<strong>in</strong>ions from authorities <strong>in</strong> certa<strong>in</strong> highhierarchycultures (e.g., not meet<strong>in</strong>g the notion of accuracy that the <strong>Standards</strong> display. <strong>The</strong>concept of ‘Uncerta<strong>in</strong>ty Avoidance’ might provide some cross-cultural psychologicalexplanation for differences <strong>in</strong> importance of reliability. For example, Hofstede (1980) talksabout Uncerta<strong>in</strong>ty Avoidance <strong>in</strong> terms of how comfortable people are with uncerta<strong>in</strong>ty orambiguity <strong>in</strong> their lives, or else, how much they value security and reliability. Jang (2000, p. 16)notes that although South Koreans “believe western systematic methodology is very useful <strong>in</strong>analyz<strong>in</strong>g <strong>in</strong>formation” (she also mentions the considerable U.S. <strong>in</strong>fluence with<strong>in</strong> the academiccommunity <strong>in</strong> South Korea), it is necessary to <strong>in</strong>volve South Korean staff to ensure receiv<strong>in</strong>greliable, valid data as the <strong>Standards</strong> call for. This quote implies that there is a “western”methodology that <strong>in</strong>variably needs adaptation when applied <strong>in</strong> non-western contexts (see Jang,2000, for advice on South Korea).
- Page 1 and 2: The Program Evaluation Standardsin
- Page 3 and 4: The editor acknowledges the valuabl
- Page 5 and 6: ForewordOn February 18-20, 2000, th
- Page 7 and 8: Hopefully, this volume will contrib
- Page 9 and 10: 5Cross-Cultural Transferabilityof T
- Page 11 and 12: Cross-Cultural Transferability of t
- Page 13 and 14: Cross-Cultural Transferability of t
- Page 15 and 16: Cross-Cultural Transferability of t
- Page 17 and 18: Cross-Cultural Transferability of t
- Page 19 and 20: Cross-Cultural Transferability of t
- Page 21 and 22: Cross-Cultural Transferability of t
- Page 23: Cross-Cultural Transferability of t
- Page 27 and 28: Cross-Cultural Transferability of t
- Page 29 and 30: Cross-Cultural Transferability of t
- Page 31: Cross-Cultural Transferability of t
- Page 34 and 35: 30 Nick L. Smith, Saviour Chircop a
- Page 36 and 37: 32 Nick L. Smith, Saviour Chircop a
- Page 38 and 39: 34 Nick L. Smith, Saviour Chircop a
- Page 40 and 41: 36 Nick L. Smith, Saviour Chircop a
- Page 42 and 43: 38 Nick L. Smith, Saviour Chircop a
- Page 44 and 45: 40 Nick L. Smith, Saviour Chircop a
- Page 46 and 47: 42 Soojung Jangyears. Due to the sc
- Page 48 and 49: 44 Soojung JangThis test-based sele
- Page 50 and 51: 46 Soojung JangTable 1. Descriptive
- Page 52 and 53: 48 Soojung JangU2 Evaluator Credibi
- Page 54 and 55: 50 Soojung Janggenerous to foreigne
- Page 56 and 57: 52 Soojung Jangfeelings. This partl
- Page 58 and 59: 54 Soojung JangReporting Informatio
- Page 60 and 61: 56 Soojung JangU4 Values Identifica
- Page 62 and 63: 58 Soojung JangReferencesBecker, C.
- Page 64 and 65: 60Standards for Evaluation:On the W
- Page 66 and 67: 62 Wolfgang BeywlThe advantage of t
- Page 68 and 69: 64 Wolfgang BeywlA more direct way
- Page 71 and 72: 67Evaluating Evaluations: Does the
- Page 73 and 74: Evaluating Evaluations 69Bussmann 1
- Page 75 and 76:
Evaluating Evaluations 71• Housin
- Page 77 and 78:
Evaluating Evaluations 73Table 1: C
- Page 79 and 80:
Evaluating Evaluations 75• Contra
- Page 81 and 82:
Evaluating Evaluations 77References
- Page 83 and 84:
Evaluating Evaluations 79Raven, J.
- Page 85 and 86:
81Evaluation StandardsRecommended b
- Page 87 and 88:
Evaluation Standard Recommended by
- Page 89 and 90:
Evaluation Standard Recommended by
- Page 91 and 92:
Evaluation Standard Recommended by
- Page 93 and 94:
Evaluation Standard Recommended by
- Page 95 and 96:
Evaluation Standard Recommended by
- Page 97 and 98:
Evaluation Standard Recommended by
- Page 99 and 100:
Evaluation Standard Recommended by
- Page 101 and 102:
Evaluation Standard Recommended by
- Page 103 and 104:
Evaluation Standard Recommended by
- Page 105 and 106:
Evaluation Standard Recommended by
- Page 107 and 108:
103AnnexThe complete list of the or
- Page 109:
Annex 105A4 Defensible Information