14 Sandy Tautpower, roles and resources as desirable and legitimate. People are socialized to comply with theirroles and the obligations related to each role. Society often penalizes people for fail<strong>in</strong>g tocomply with the obligations imparted by their social status (Berry et al., 1997, Vol. 3, p. 79).Thus, paradoxically, what the <strong>Standards</strong> list as Common Errors may serve as Guidel<strong>in</strong>es <strong>in</strong> highhierarchy cultures, e.g., when stat<strong>in</strong>g: "Assum<strong>in</strong>g that persons <strong>in</strong> leadership or decision-mak<strong>in</strong>groles are the only, or most important, stakeholders." (Jo<strong>in</strong>t Committee, 1994, p. 26. For example,for South Korea, Jang (2000, p. 10) po<strong>in</strong>ts out that “the project focus is likely to be determ<strong>in</strong>edby the specific client group that supports and controls the project.” Throughout her analysis ofthe applicability of the <strong>Standards</strong> <strong>in</strong> South Korean society, Jang stresses the importance of thefocus on the top client group (see Jang, 2000, pp. 10) and she concludes that “evaluators […]need to meet only the immediate stakeholder’s expectations” (p. 24).U2 Evaluator Credibility <strong>The</strong> persons conduct<strong>in</strong>g the evaluation shouldbe both trustworthy and competent to perform the evaluation, so that the evaluationf<strong>in</strong>d<strong>in</strong>gs achieve maximum credibility and acceptance.<strong>The</strong> second Utility Standard U2, Evaluator Credibility, also touches upon culturallyrelevant value dimensions. As po<strong>in</strong>ted out before by Smith et al. (1994, p. 7), the concept of aprofessional differs across cultures (e.g., between Malta, India, and the U.S.). <strong>The</strong>y state that “<strong>in</strong>the U.S., a professional is someone who […] has acquired a certa<strong>in</strong> expertise [which] isofficially acknowledged through certification and licensure […],” whereas <strong>in</strong> Malta, “the notionof professional is much looser […].” In India, “‘Professionalism’ is not a common word and tothe orthodox or older generation it connotes a lack of love for the job […]. Professionalism hasnoth<strong>in</strong>g to do with the level of one’s expertise” (p. 7). Consequently, the notion of ‘competence’as specified <strong>in</strong> the <strong>Program</strong> <strong>Evaluation</strong> <strong>Standards</strong> (see Jo<strong>in</strong>t Committee, 1994, p. 31) is not aprerequisite for an evaluator’s credibility across all cultures. <strong>The</strong> characteristics that causeclients to judge the professional evaluator as competent and trustworthy differ across cultures.For example, cross-cultural organizational psychology uses the pr<strong>in</strong>ciple of seniority to describedifferences <strong>in</strong> authority held by members of an organization (Berry et al., 1997, Vol. 3, p. 379).Accord<strong>in</strong>g to this pr<strong>in</strong>ciple, older people are per se attributed a higher degree of competence <strong>in</strong>certa<strong>in</strong> cultures, regardless of other factors also <strong>in</strong>fluenc<strong>in</strong>g competence, e.g., level of education(see Jang 2000, p. 26). <strong>The</strong> social status of a person (often positively correlated with age) alsoaffects the attribution of competence and trustworth<strong>in</strong>ess <strong>in</strong> certa<strong>in</strong> cultures, contrary to a moreachievement-oriented, <strong>in</strong>dividualistic approach to professional worth<strong>in</strong>ess. <strong>The</strong> latter is conveyed<strong>in</strong> the <strong>Standards</strong> when stat<strong>in</strong>g that "evaluators are credible to the extent that they exhibit thetra<strong>in</strong><strong>in</strong>g, technical competence, substantive knowledge, experience, <strong>in</strong>tegrity, public relationsskills, and other characteristics considered necessary by clients […]" (Jo<strong>in</strong>t Committee, 1994, p.31). For example, Jang (2000, p. 11) notes that the well-known “name” of the evaluator is thebasis for be<strong>in</strong>g considered competent by South Korean clients. As another example, an Indianevaluator (see Hendricks & Conner, 1995, p. 83) describes his culture as one <strong>in</strong> which“personalized relationships and connections matter more than public <strong>in</strong>terest,” so that “a nexus
Cross-Cultural Transferability of the <strong>Program</strong> <strong>Evaluation</strong> <strong>Standards</strong> 15develops where evaluation is a means to bestow favors to each other irrespective of their levelsof competence.”U3 Information Scope and Selection Information collected should be broadlyselected to address pert<strong>in</strong>ent questions about the program and be responsive to theneeds and <strong>in</strong>terest of clients and other specified stakeholders.Standard U3, Information Scope and Selection, mentions multiple stakeholders, "each ofwhom should have an opportunity to provide <strong>in</strong>put to the evaluation design and have a claim foraccess to the results" (Jo<strong>in</strong>t Committee, 1994, p. 37). As po<strong>in</strong>ted out before, compliance with thisStandard is not necessarily desirable <strong>in</strong> a hierarchically structured society. <strong>The</strong> client and theevaluator might not f<strong>in</strong>d it important to extend <strong>in</strong>volvement to other than the s<strong>in</strong>gle most<strong>in</strong>fluential stakeholder group because other groups may not or should not <strong>in</strong>fluence theevaluation process and results (see Jang, 2000, p. 10). In addition, group-orientation and loyaltytoward authorities may forbid certa<strong>in</strong> stakeholder groups to voice their op<strong>in</strong>ions, contrary to thedemocratic approach generally evident <strong>in</strong> the <strong>Standards</strong> (Jo<strong>in</strong>t Committee, 1994). Also, as Jangobserves, “Koreans […] are not as experienced as Americans <strong>in</strong> express<strong>in</strong>g their personalop<strong>in</strong>ions” (2000, p. 18). <strong>The</strong> author attributes this phenomenon to “the cultural norm of<strong>in</strong>direction” (p. 18). In addition, there is a high tendency to perceive group consensus (falsegroup consensus effect) <strong>in</strong> collectivist cultures which further limits the <strong>in</strong>formation scope of theevaluation <strong>in</strong> these cultures (see Berry et al., 1997, Vol. 3, chapter 1, p. 58; Jang, 2000, p. 18, foran example from South Korea). Contrary to <strong>in</strong>dividualistic societies (e.g., Australia) where<strong>in</strong>group members are expected to act as <strong>in</strong>dividuals even if it means go<strong>in</strong>g aga<strong>in</strong>st the <strong>in</strong>group, <strong>in</strong>collectivist cultures (e.g., Indonesia, see Noesjirwan, 1978) the members of the <strong>in</strong>group shouldadapt to the group, ameliorat<strong>in</strong>g its unity (see Berry et al., 1997, Vol. 3, p. 143).U4 Values Identification <strong>The</strong> perspectives, procedures, and rationaleused to <strong>in</strong>terpret the f<strong>in</strong>d<strong>in</strong>gs should be carefully described, so that the bases forvalue judgments are clear.Standard U4, Values Identification, is an example of a culturally sensitive Standard. Itcalls for the acknowledgment and description of contextually different value bases for<strong>in</strong>terpretation of obta<strong>in</strong>ed <strong>in</strong>formation. "Value is the root term <strong>in</strong> evaluation; and valu<strong>in</strong>g […] isthe fundamental task <strong>in</strong> evaluations" (Jo<strong>in</strong>t Committee, 1994, p. 43). Values differ across<strong>in</strong>dividuals as well as across cultures (see above); therefore "it seems doubtful that any particularprescription for arriv<strong>in</strong>g at value judgments is consistently the best <strong>in</strong> all evaluative contexts"(Jo<strong>in</strong>t Committee, 1994, p. 43).
- Page 1 and 2: The Program Evaluation Standardsin
- Page 3 and 4: The editor acknowledges the valuabl
- Page 5 and 6: ForewordOn February 18-20, 2000, th
- Page 7 and 8: Hopefully, this volume will contrib
- Page 9 and 10: 5Cross-Cultural Transferabilityof T
- Page 11 and 12: Cross-Cultural Transferability of t
- Page 13 and 14: Cross-Cultural Transferability of t
- Page 15 and 16: Cross-Cultural Transferability of t
- Page 17: Cross-Cultural Transferability of t
- Page 21 and 22: Cross-Cultural Transferability of t
- Page 23 and 24: Cross-Cultural Transferability of t
- Page 25 and 26: Cross-Cultural Transferability of t
- Page 27 and 28: Cross-Cultural Transferability of t
- Page 29 and 30: Cross-Cultural Transferability of t
- Page 31: Cross-Cultural Transferability of t
- Page 34 and 35: 30 Nick L. Smith, Saviour Chircop a
- Page 36 and 37: 32 Nick L. Smith, Saviour Chircop a
- Page 38 and 39: 34 Nick L. Smith, Saviour Chircop a
- Page 40 and 41: 36 Nick L. Smith, Saviour Chircop a
- Page 42 and 43: 38 Nick L. Smith, Saviour Chircop a
- Page 44 and 45: 40 Nick L. Smith, Saviour Chircop a
- Page 46 and 47: 42 Soojung Jangyears. Due to the sc
- Page 48 and 49: 44 Soojung JangThis test-based sele
- Page 50 and 51: 46 Soojung JangTable 1. Descriptive
- Page 52 and 53: 48 Soojung JangU2 Evaluator Credibi
- Page 54 and 55: 50 Soojung Janggenerous to foreigne
- Page 56 and 57: 52 Soojung Jangfeelings. This partl
- Page 58 and 59: 54 Soojung JangReporting Informatio
- Page 60 and 61: 56 Soojung JangU4 Values Identifica
- Page 62 and 63: 58 Soojung JangReferencesBecker, C.
- Page 64 and 65: 60Standards for Evaluation:On the W
- Page 66 and 67: 62 Wolfgang BeywlThe advantage of t
- Page 68 and 69:
64 Wolfgang BeywlA more direct way
- Page 71 and 72:
67Evaluating Evaluations: Does the
- Page 73 and 74:
Evaluating Evaluations 69Bussmann 1
- Page 75 and 76:
Evaluating Evaluations 71• Housin
- Page 77 and 78:
Evaluating Evaluations 73Table 1: C
- Page 79 and 80:
Evaluating Evaluations 75• Contra
- Page 81 and 82:
Evaluating Evaluations 77References
- Page 83 and 84:
Evaluating Evaluations 79Raven, J.
- Page 85 and 86:
81Evaluation StandardsRecommended b
- Page 87 and 88:
Evaluation Standard Recommended by
- Page 89 and 90:
Evaluation Standard Recommended by
- Page 91 and 92:
Evaluation Standard Recommended by
- Page 93 and 94:
Evaluation Standard Recommended by
- Page 95 and 96:
Evaluation Standard Recommended by
- Page 97 and 98:
Evaluation Standard Recommended by
- Page 99 and 100:
Evaluation Standard Recommended by
- Page 101 and 102:
Evaluation Standard Recommended by
- Page 103 and 104:
Evaluation Standard Recommended by
- Page 105 and 106:
Evaluation Standard Recommended by
- Page 107 and 108:
103AnnexThe complete list of the or
- Page 109:
Annex 105A4 Defensible Information