30 Nick L. Smith, Saviour Chircop and Prachee MukherjeeIn this article we present prelim<strong>in</strong>ary considerations on two questions related to these crossculturalevaluation concerns: to what extent might the Jo<strong>in</strong>t Committee (1981) standards be usedas a basis for develop<strong>in</strong>g comparable, culturally relevant standards for <strong>in</strong>digenous use, and <strong>in</strong>what ways might the Jo<strong>in</strong>t Committee (1981) standards be modified for use by U.S. evaluators <strong>in</strong>cross-cultural sett<strong>in</strong>gs.Initially we outl<strong>in</strong>ed a straightforward development process: exam<strong>in</strong>e the Jo<strong>in</strong>t Committee(1981) standards from a particular cultural perspective and (a) delete those standards that appearirrelevant or <strong>in</strong>appropriate, (b) modify any standards to conform with the new cultural norms,and (c) add any new standards that would be required, so that the f<strong>in</strong>al set would be a culturallyrelevant counterpart to the orig<strong>in</strong>al U.S. standards. We selected two disparate cultures to use <strong>in</strong>contrast with the U.S. culture with<strong>in</strong> which the Jo<strong>in</strong>t Committee (1981) standards weredeveloped: the culture of a small, Mediterranean develop<strong>in</strong>g country–Malta, and the culture of alarge, eastern, developed country–India. Two of the co-authors of this article are from Malta andIndia respectively, have studied and practiced evaluation <strong>in</strong> the U.S., and are <strong>in</strong>timately familiarwith the Jo<strong>in</strong>t Committee (1981) standards. In order to understand better the nature of culturallyrelevant evaluation standards, then, we <strong>in</strong>itially worked to develop versions of the standards thatcould be used by <strong>in</strong>digenous evaluators <strong>in</strong> Malta and India, and/or by U.S. evaluators work<strong>in</strong>gwith<strong>in</strong> the Maltese or Indian cultures.We soon came to realize that this seem<strong>in</strong>gly straightforward task actually <strong>in</strong>volved a k<strong>in</strong>dof “l<strong>in</strong>guistic translation” strategy <strong>in</strong> which (a) a given U.S. standard had to be analyzed <strong>in</strong> termsof its underly<strong>in</strong>g U.S. values, norms, laws, and context, (b) a translation to counterpart values,norms, laws, and context <strong>in</strong> the second culture had to be made, and (c) a “new” counterpartstandard relevant to the second culture had to be envisioned based on these translated values,norms, etc. This strategy had the benefits of uncover<strong>in</strong>g a deeper understand<strong>in</strong>g of the Jo<strong>in</strong>tCommittee (1981) standards as a reflection of American culture and liberal democracy, and ofheighten<strong>in</strong>g our awareness of some of the cultural bases of differences <strong>in</strong> evaluation practice <strong>in</strong>cross-cultural sett<strong>in</strong>gs.<strong>The</strong> full development of <strong>in</strong>dependent collections of culturally relevant evaluation standardsis, of course, a highly complicated task. Through our exploratory efforts us<strong>in</strong>g this “standardstranslation” process, however, we did achieve some important <strong>in</strong>sights <strong>in</strong>to the relationshipsamong culture, standards, and evaluation practice. Before proceed<strong>in</strong>g to the results of ouranalysis, it is important that the reader have some appreciation of the historical and evaluationcontexts of the two non-U.S. cultures used <strong>in</strong> this study.<strong>The</strong> Historical and <strong>Evaluation</strong> Contexts of Malta and IndiaBoth Malta and India were once British colonies. Malta is a small country with a localidentity which experienced a friendly transition to <strong>in</strong>dependence <strong>in</strong> 1964. India is a very largecountry with many sub-culture identities. Its <strong>in</strong>dependence was achieved <strong>in</strong> 1947 <strong>in</strong> a much moreadversarial manner.
Considerations on the Development of Culturally Relevant <strong>Evaluation</strong> <strong>Standards</strong> 31Malta is a small Mediterranean island (122 square miles) with a population of 350,000. It isone of the top countries with<strong>in</strong> the category of “develop<strong>in</strong>g countries” as listed by the WorldBank and <strong>International</strong> Monetary Fund. <strong>The</strong> Island has a history that dates back more than 5000years, and has been dom<strong>in</strong>ated by ever group seek<strong>in</strong>g control of the Mediterranean. <strong>The</strong> survivalof its people has depended on their ability to operate simultaneously with<strong>in</strong> European and NorthAfrican cultural milieus. Due to its history of colonization, <strong>in</strong>terpersonal relationships are highlyimportant, k<strong>in</strong>ship and friendship ties are very strong <strong>in</strong> Malta. <strong>The</strong> political position of theperson one is speak<strong>in</strong>g with is of considerable concern, and communication is most rapid along“friend-of-friends” patterns. To be effective, the evaluator must be able to make use of thesefriends-of-friends networks and to refra<strong>in</strong> from ask<strong>in</strong>g for <strong>in</strong>formation <strong>in</strong> a way that the<strong>in</strong>formant could be identified.Chircop (1987) describes the role of evaluation <strong>in</strong> Malta as follows:Formalized evaluation <strong>in</strong> Malta has largely been <strong>in</strong>stigated by foreign needs and demands.With<strong>in</strong> a colonial structure, the representatives of the powers that be were always requiredto send reports to their superiors. Evidence of these reports exists for 2000 years. <strong>The</strong>reports by the various commissions and governors sent to the British Crown are perhaps themost recent and preserved documents. When the Island became <strong>in</strong>dependent, such reportswere conducted by foreigners for the purpose of aid agreements with national or<strong>in</strong>ternational <strong>in</strong>stitutions. <strong>Evaluation</strong>s have also been conducted by foreign <strong>in</strong>dustrialconcerns. Thus one might say that formalized evaluation, though existent <strong>in</strong> Malta, islargely a foreign concern (Chircop, p.8).With a population of over 700 million, India is the seventh largest country <strong>in</strong> the world,three-fifths the size of the U.S. It has a long history that dates back to 1500 B.C. and has beendom<strong>in</strong>ated by Islamic and British powers for more than 700 years. <strong>The</strong> Indian tradition ischaracterized by cultural heterogeneity and religious tolerance. As laid down <strong>in</strong> the Indianconstitution, India is a secular democracy.India’s political structure consists of a powerful central government; any state governmentwhich happens to be headed by an opposition party will have a most difficult time ga<strong>in</strong><strong>in</strong>gapproval for its projects. Although India’s constitution provide for a multi-party system, theCongress party has historically dom<strong>in</strong>ated all other parties lack<strong>in</strong>g a national majority. InNovember 1989, the Congress party failed to w<strong>in</strong> the national elections, only the second times<strong>in</strong>ce India’s <strong>in</strong>dependence <strong>in</strong> 1947.<strong>The</strong> major problems of India are over-population, poverty, and unemployment. <strong>The</strong>seproblems have to be kept <strong>in</strong> m<strong>in</strong>d <strong>in</strong> understand<strong>in</strong>g what works and what does not work <strong>in</strong> India;what is acceptable and what is not. <strong>The</strong> “culture” of India changes and reacts to these problems,and it is not possible to understand India’s culture <strong>in</strong>dependently of the context these problemscreate.
- Page 1 and 2: The Program Evaluation Standardsin
- Page 3 and 4: The editor acknowledges the valuabl
- Page 5 and 6: ForewordOn February 18-20, 2000, th
- Page 7 and 8: Hopefully, this volume will contrib
- Page 9 and 10: 5Cross-Cultural Transferabilityof T
- Page 11 and 12: Cross-Cultural Transferability of t
- Page 13 and 14: Cross-Cultural Transferability of t
- Page 15 and 16: Cross-Cultural Transferability of t
- Page 17 and 18: Cross-Cultural Transferability of t
- Page 19 and 20: Cross-Cultural Transferability of t
- Page 21 and 22: Cross-Cultural Transferability of t
- Page 23 and 24: Cross-Cultural Transferability of t
- Page 25 and 26: Cross-Cultural Transferability of t
- Page 27 and 28: Cross-Cultural Transferability of t
- Page 29 and 30: Cross-Cultural Transferability of t
- Page 31: Cross-Cultural Transferability of t
- Page 36 and 37: 32 Nick L. Smith, Saviour Chircop a
- Page 38 and 39: 34 Nick L. Smith, Saviour Chircop a
- Page 40 and 41: 36 Nick L. Smith, Saviour Chircop a
- Page 42 and 43: 38 Nick L. Smith, Saviour Chircop a
- Page 44 and 45: 40 Nick L. Smith, Saviour Chircop a
- Page 46 and 47: 42 Soojung Jangyears. Due to the sc
- Page 48 and 49: 44 Soojung JangThis test-based sele
- Page 50 and 51: 46 Soojung JangTable 1. Descriptive
- Page 52 and 53: 48 Soojung JangU2 Evaluator Credibi
- Page 54 and 55: 50 Soojung Janggenerous to foreigne
- Page 56 and 57: 52 Soojung Jangfeelings. This partl
- Page 58 and 59: 54 Soojung JangReporting Informatio
- Page 60 and 61: 56 Soojung JangU4 Values Identifica
- Page 62 and 63: 58 Soojung JangReferencesBecker, C.
- Page 64 and 65: 60Standards for Evaluation:On the W
- Page 66 and 67: 62 Wolfgang BeywlThe advantage of t
- Page 68 and 69: 64 Wolfgang BeywlA more direct way
- Page 71 and 72: 67Evaluating Evaluations: Does the
- Page 73 and 74: Evaluating Evaluations 69Bussmann 1
- Page 75 and 76: Evaluating Evaluations 71• Housin
- Page 77 and 78: Evaluating Evaluations 73Table 1: C
- Page 79 and 80: Evaluating Evaluations 75• Contra
- Page 81 and 82: Evaluating Evaluations 77References
- Page 83 and 84: Evaluating Evaluations 79Raven, J.
- Page 85 and 86:
81Evaluation StandardsRecommended b
- Page 87 and 88:
Evaluation Standard Recommended by
- Page 89 and 90:
Evaluation Standard Recommended by
- Page 91 and 92:
Evaluation Standard Recommended by
- Page 93 and 94:
Evaluation Standard Recommended by
- Page 95 and 96:
Evaluation Standard Recommended by
- Page 97 and 98:
Evaluation Standard Recommended by
- Page 99 and 100:
Evaluation Standard Recommended by
- Page 101 and 102:
Evaluation Standard Recommended by
- Page 103 and 104:
Evaluation Standard Recommended by
- Page 105 and 106:
Evaluation Standard Recommended by
- Page 107 and 108:
103AnnexThe complete list of the or
- Page 109:
Annex 105A4 Defensible Information