A Note on Usefulness of Geometrical Crossover for Numerical ...

A <strong>Note</strong> on Usefulness of Geometrical Crossover for NumericalZbigniew MichalewiczDepartment of Computer ScienceUniversity of North CarolinaCharlotte, NC 28223zbyszek@uncc.eduOptimization ProblemsMaciej MichalewiczInstitute of Computer SciencePolish Academy of Sciencesul. Ordona 2101-237 Warsaw, Polandmichalew@ipipan.waw.plGirish NazhiyathDepartment of Computer ScienceUniversity of North CarolinaCharlotte, NC 28223gnazhiya@uncc.eduAbstractNumerical optimization problems enjoy a signicantpopularity in evolutionary computationcommunity; all major evolutionary techniques (geneticalgorithm, evolution strategies, evolutionaryprogramming) have been applied to these problems.However, many of these techniques (as well asother, classical optimization methods) have dicultiesin solving some real-world problems whichinclude non-trivial constraints. For such problems,very often the global solution lies on theboundary of the feasible region. Thus it is importantto investigate some problem-specic operators,which search this boundary in an ecientway.In this study we discuss a new experimental evidenceon usefulness of so-called geometrical crossover,which might be used for a boundary searchfor particular problems. This operator enhancesalso the eectiveness of evolutionary algorithms(based on oating point representation) in a signicantway.1 IntroductionFor many years, most evolutionary techniques were evaluatedand compared with each other in the domain offunction optimization. It seems also that the domainof function optimization will remain the primary testbedfor many new comparisons and new features of variousalgorithms. In particular, numerical optimizationproblems enjoy a signicant popularity in evolutionarycomputation community; all major evolutionary techniques(genetic algorithms, evolution strategies, evolutionaryprogramming) can be applied to these problems.However, in this study we depart from dierences betweenvarious evolutionary techniques for numerical optimizationproblems, and discuss rather some experimentalevidence on usefulness of a new operator, so-called geometricalcrossover. This operator seems to enhance theeectiveness of evolutionary algorithms (based on oatingpoint representation) in a signicant way; later inthe paper we compare it with a better known arithmeticalcrossover on a few test cases.The geometrical crossover emerged as a problem-specicoperator for one particular constrained problem. Ina similar way, we can construct (for other constrainedproblems) various crossover operators (e.g., sphere crossover;see Conclusions) and analyse their performance.Thus this study indicates another possible way for constrainthandling in evolutionary methods, based on asearch for a global solution on the boundary of the feasibleregion. Some other heuristic techniques (e.g., tabusearch) recognized recently the importance of such searchby investigating strategic oscillation (see Kelly et al. 1993,Glover and Kochenberger, 1995).The paper is organized as follows. The followingtwo sections survey briey several operators and a fewconstraint-handling techniques for numerical optimizationproblems, which have emerged in evolutionary computationtechniques (oating point representation) overthe years. Section 4 presents a test case of a constrainedoptimization problem, which proved to be extremely dif-cult for most optimization methods. Section 5 discussesa special, problem-specic evolutionary system, which

was \tuned" towards this particular problem; the systemincorporates a new operator, which we call geometricalcrossover. There is some experimental evidence on a generalusefulness of this operator, which outperforms a betterknown arithmetical crossover; this is presented in Section6. We conclude the paper by providing an exampleof another problem-specic crossover, sphere crossover,and indicate its usefulness on another test case.2 Numerical Optimization and OperatorsMost evolutionary algorithms use vectors of oating pointnumbers for their chromosomal representations. For suchrepresentation, many operators have been proposed duringthe last 30 years. We discuss them briey in turn.The most popular mutation operator is Gaussian mutation,which modied all components of the solutionvector ~x = hx 1 ; : : : ; x n i by adding a random noise:~x t+1 = ~x t + N(0; ~),N(0; ~) is a vector of independent random Gaussian numberswith a mean of zero and standard deviations ~.Such a mutation is used in evolution strategies (Back etal. 1991) and evolutionary programming (Fogel 1992).(One of the dierences between these techniques lies inadjusting vector of standard deviations ~.)Other types of mutations include non-uniform mutation,wherex t+1k= xtk+ 4(t; right(k) x k ) if r is 0x t k 4(t; x k left(k)) if r is 1for k = 1; : : : ; n (r is a random binary digit). The function4(t; y) returns a value in the range [0; y] such thatthe probability of 4(t; y) being close to 0 increases ast increases (t is the generation number). This propertycauses this operator to search the space uniformly initially(when t is small), and very locally at later stages.In experiments reported in Michalewicz et al. (1994),the following function was used:4(t; y) = y r (1tT )b ;where r is a random number from [0::1], T is the maximalgeneration number, and b is a system parameterdetermining the degree of non{uniformity.It is also possible to experiment with a uniform mutation,which changes a single component of the solutionvector; e.g., if ~x t = (x 1 ; : : :; x k ; : : : ; x q ), then ~x t+1 =(x 1 ; : : : ; x 0 k ; : : : ; x q), where x 0 kis a random value (uniformprobability distribution) from the domain of variable x k .A special case of uniform mutation is boundary mutation,where x 0 k is either left of right boundary of the domainof x k .There are a few interesting crossover operators. Therst one, uniform crossover (known also as discrete crossoverin evolution strategies; Schwefel 1981), works as follows.Two parents,~x 1 = hx 1 1 ; : : : ; x1 n i and ~x2 = hx 2 1 ; : : : ; x2 n i,produce an ospring,~x = hx q11 ; : : : ; xqn n i,where q i = 1 or q i = 2 with equal probability for alli = 1; : : :; n (the second ospring is created by settingq i := 3 q i ). Of course, the uniform crossover is a generalizationof 1-point, 2-point, and multi-point crossovers.Another possibility is arithmetical crossover. Heretwo vectors, ~x 1 and ~x 2 produce two ospring, ~x 3 and ~x 4 ,which are a linear combination of their parents, i.e.,~x 3 = a ~x 1 + (1 a) ~x 2 and~x 4 = (1 a) ~x 1 + a ~x 2 .Such a crossover was called also a guaranteed averagecrossover (Davis, 1989, when a = 1=2), and an intermediatecrossover (Schwefel, 1981).There is also an interesting heuristic crossover proposedby Wright (1991); it is a unique crossover for thefollowing reasons: (1) it uses values of the objective functionin determining the direction of the search, (2) itproduces only one ospring, and (3) it may produce noospring at all. The operator generates a single ospring~x 3 from two parents, ~x 1 and ~x 2 according to the followingrule:~x 3 = r (~x 2 ~x 1 ) + ~x 2 ,where r is a random number between 0 and 1, and theparent ~x 2 is not worse than ~x 1 , i.e., f(~x 2 ) f(~x 1 ) formaximization problems and f(~x 2 ) f(~x 1 ) for minimizationproblems.Muhlenbein and Voigt (1995) investigated the propertiesof a recombination operator, called gene pool recombination(default recombination mechanism in evolutionstrategies, see Schwefel, 1981), where the genesare randomly picked from the gene pool dened by theselected parents. An interesting aspect of this operatoris that it allows so-called orgies: several parents in producingan ospring. Such a multi-parent crossover wasalso investigated also by Eiben et al. (1994) in the contextof combinatorial optimization. Renders and Bersini(1994) experimented with simplex crossover for numericaloptimization problems; this crossover involves computingcentroid of group of parents and moving fromthe worst individual beyond the centroid point. Also,Glover's (1977) scatter search techniques propose the useof multiple parents.3 Constraint-Handling MethodsDuring the last two years several methods were proposedfor handling constraints by genetic algorithms for numericaloptimization problems. Most of them are based on2

the concept of penalty functions, which penalize unfeasiblesolutions, i.e., f(X);if X 2 Feval(X) =f(X) + penalty(X); if X 2 S F;where the set S R n denes the search space and the setF S denes a feasible search space, penalty(X) is zero,if no violation occurs, and is positive, otherwise (in therest of this section we assume minimization problems).In most methods a set of functions f j (1 j m) isused to construct the penalty; the function f j measuresthe violation of the j-th constraint in the following way: maxf0; gj (X)g; if 1 j qf j (X) =jh j (X)j; if q + 1 j m:However, these methods dier in many important details,how the penalty function is designed and appliedto unfeasible solutions.The most severe penalty is a death penalty; somemethod rejects unfeasible individuals. Such a methodhas been used by by many techniques, e.g., evolutionstrategies (Back et al. 1991) and simulated annealing.Some methods use static penalties; for example, amethod proposed by Homaifar, Lai, and Qi (1994) assumesthat for every constraint we establish a familyof intervals which determine appropriate penalty coecient.It works by creating (for each constraint) several(`) levels of violation; creating a penalty coecient R ij(for each level of violation and for each constraint; higherlevels of violation require larger values of this coecient;starting with a random population of individuals (feasibleor unfeasible); and evolving the population, wherethe evaluation function used iseval(X) = f(X) + P mj=1 R ijf 2 j (X).Some other methods apply dynamic penalties. Forexample, in the method proposed by Joines and Houck(1994) individuals are evaluated (at the iteration t) bythe following formula:eval(X) = f(X) + (C t) P mj=1 f j (X),where C, and are constants. A reasonable choice forthese parameters is C = 0:5, = = 2.Also, Michalewicz and Attia (1994) experimented withthe procedure, where the algorithm maintains feasibilityof all linear constraints using a set of closed operators,which convert a feasible solution (feasible in terms of linearconstraints only) into another feasible solution. Atevery iteration the algorithm considers active constraintsonly, the pressure on unfeasible solutions is increased dueto the decreasing values of temperature .It is also possible to incorporate adaptive penalties;Bean and Hadj-Alouane (1992) and Smith and Tate (1993)experimented with them. Bean and Hadj-Alouane (1992)change the penalty coecients on the basis of the numberof feasible and infeasible individuals in the last k generations,whereas in the Smith and Tate (1993) approachpenalty measure depends on the number of violated constraints,the best feasible objective function found, andthe best objective function value found.Yet another approach was proposed recently by LeRiche et al. (1995). The authors designed a (segregated)genetic algorithm which uses two values of penalty parameters(for each constraint) instead of one; these twovalues aim at achieving a balance between heavy andmoderate penalties by maintaining two subpopulationsof individuals. The population is split into two cooperatinggroups, where individuals in each group are evaluatedusing either one of the two penalty parameters.Additional method was proposed by Schoenauer andXanthakis (1993) The method requires a linear order ofall constraints which are processed in turn. At iterationj, solutions that do not satisfy one of the 1st, 2nd, ...,or (j 1)-th constraint are eliminated from the populationand the stop criterion is the satisfaction of thej-th constraint by the ip threshold percentage of thepopulation. In the last step the algorithm optimizes theobjective function, rejecting unfeasible individuals.Another method was developed by Powell and Skolnick(1993). The method is a classical penalty methodwith one notable exception. Each individual is evaluatedby the formula:eval(X) = f(X) + r P mj=1 f j(X) + (t; X),where r is a constant; however, there is also a component(t; X). This is an additional iteration dependentfunction which inuences the evaluations of unfeasiblesolutions. The point is that the method distinguishes betweenfeasible and unfeasible individuals by adopting anadditional heuristic rule (suggested earlier by Richardsonet al. 1989): for any feasible individual X and anyunfeasible individual Y : eval(X) < eval(Y ), i.e., anyfeasible solution is better than any unfeasible one.One of the most recent methods (Genocop III; seeMichalewicz and Nazhiyath 1995) incorporates the originalGenocop system (Michalewicz et al. 1994), butalso extends it by maintaining two separate populations,where a development in one population inuences evaluationsof individuals in the other population. The rstpopulation consists of so-called search points from Swhich satisfy linear constraints of the problem (as in theoriginal Genocop system). The feasibility (in the senseof linear constraints) of these points is maintained, asbefore, by specialized operators. The second populationconsists of so-called reference points from F; these pointsare fully feasible, i.e., they satisfy all constraints. Referencepoints R, being feasible, are evaluated directly bythe objective function (i.e., eval(R) = f(R)). On theother hand, unfeasible search points are \repaired" forevaluation.3

For an experimental comparison of some of these methodson a few test cases, see Michalewicz (1995).4 The Test CaseAn interesting constrained numerical optimization testcase emerged recently; the problem (Keane, 1994) is tomaximize a function:P ni=1f(~x) = jcos4 (x i ) 2 Q ni=1 cos2 (x i )pP n j;i=1 ix2 iwhereQ ni=1 x i 0:75, (1)P ni=1 x i 7:5n, (2)and 0 x i 10 for 1 i n.The problem has two constraints; the function f is nonlinearand its global maximum is unknown.10.80.60.40.201008060402000Figure 2: The graph of function f for n = 2. Infeasiblesolutions were assigned value zero2040608010012108642025201510500Figure 1: The graph of function f for n = 2To illustrate some potential diculties of solving thistest case, a few graphs (for case of n = 2) are displayedin Figures 1 { 4. Figure 1 gives a general overview of theobjective function f: the whole landscape seems to berelatively at except for a sharp peek around the point(0; 0). Figures 2 and 3 incorporate the active constraint:infeasible solutions were assigned a value of zero. In thatcase the objective function f takes values from the rangeh0; 1i; because of the scaling, the landscape is more visible.Figure 3 displays only the area of interest (i.e., thearea of global optimum). Figure 4 presents a side viewof the landscape along one of the axis. Again, the onlyfeasible part of the landscape is displayed.All methods (described in section 3) were tried onthis test case with quite poor results. As Keane (1994)noted:510152025\I am currently using a parallel GA with 12bitbinary encoding, crossover, inversion, mutation,niche forming and a modied Fiacco-McCormickconstraint penalty function to tackle this. For n =20 I get values like 0.76 after 20,000 evaluations."We obtained the best results from Genocop III; however,the results varied from run to run (between 0.75 and 0.80,case of n = 20).5 A New System and the GeometricalCrossoverIt seems that majority of constraint-handling methodshave serious diculties in returning a high quality solutionfor the above problem. It might be possible, however,to build a dedicated system for this particular testcase, which would incorporate the problem specic knowledge.This knowledge can emerge from analysis of theobjective function f and the constraints; thus we assumethat (a) the domain for all variables is h0:1; 10:0i, (b) constraint(1) is active at the global optimum, and (c) theconstraint (2) is not.Now it is possible to develop an evolutionary systemwhich would search just the surface dened by the rstconstraint:Q ni=1 x i = 0:75.Thus the search space is greatly reduced; we consideronly points which satisfy the above equation. Such asystem would start from a population of feasible points,i.e., points, for which a product of all coordinates is equalto 0.75. It is relatively easy to develop such initializationroutine:4

0.90.810.70.80.60.60.50.40.40.20.30010203040 051015200.20.10100500 20 40 60 80 100Figure 3: The graph of function f for n = 2. Infeasiblesolutions were assigned value zero; only the corneraround the global optimum is shownfor i = 1 to n dobeginif i is evenx i 1 = random (0.1, 10)x i = 1=x i 1endif n is oddx n = 0:75elsefor some random ix i = 0:75 x iIt is important to note, that the above initializationroutine is not signicant for the performance of any system;it just ensures that all points in the population areat the boundary between feasible and infeasible regions.Many other methods were tried with such initialized populationsand gave poor results. Also, the value of the bestindividual in the population initialized in such a way isaround 0.30.The geometrical crossover takes two parents and producesa single ospring; for parents ~x 1 and ~x 2 the ospring~x 3 is~x 3 = p p h x 1 1 x2 1 ; : : : ; x 1 n x2 ni. (3)<strong>Note</strong>, that the ospring ~x 3 also lies on the boundary offeasible region:Q ni=1 x3 i = 0:75.Of course, it is an easy task to generalize the above geometricalcrossover into~x 3 = h(x 1 1) (x 2 1) (1 ) ; : : :; (x 1 n) (x 2 n) (1 ) i, (4)Figure 4: The vertical view of the function f for n = 2for 0 1. Also it is possible to include several (say,k) parents:~x k+1 = h(x 1 1) 1 (x 2 1) 2 : : : (x k 1) k ; : : : ;(x 1 n) 1 (x 2 n) 2 : : : (x k n) k i, (5)where 1 +: : :+ k = 1. However, in the experiments reportedin this paper, we limited ourselves to the simplestgeometrical crossover (3).The task of designing a feasibility-preserving mutationis relatively simple; if the i-th component is selectedfor mutation, the following algorithm is executed:determine random j, 1 j n, j 6= iselect q such that:0:1 x i q 10:0 and0:1 x j =q 10:0x i = x i qx j = x j =qA simple evolutionary algorithm (200 lines of code!)with geometrical crossover and problem-specic mutationgave an outstanding results. For the case n = 20the system reached the value of 0.80 in less than 4,000generations (with population size of 30, probability ofcrossover p c = 1:0, and probability of mutation p m =0:06) in all runs. The best value found (namely 0.803553)was better than the best values of any method discussedearlier, whereas the worst value found was 0.802964. Similarly,for n = 50, all results (in 30,000 generations) werebetter than 0.83 (with the best of 0.8331937);~x = (6.28006029, 3.16155291, 3.15453815, 3.14085174,3.12882447, 3.11211085, 3.10170507, 3.08703685,3.07571769, 3.06122732, 3.05010581, 3.03667951,3.02333045, 3.00721049, 2.99492717, 2.97988462,2.96637058, 2.95589066, 2.94427204, 2.92796040,5

0.40970641, 2.90670991, 0.46131119, 0.48193336,0.46776962, 0.43887550, 0.45181099, 0.44652876,0.43348753, 0.44577143, 0.42379948, 0.45858049,0.42931050, 0.42928645, 0.42943302, 0.43294361,0.42663351, 0.43437257, 0.42542559, 0.41594154,0.43248957, 0.39134723, 0.42628688, 0.42774364,0.41886297, 0.42107263, 0.41215360, 0.41809589,0.41626775, 0.42316407),It was interesting to note the importance of geometricalcrossover. With xed population size (kept constantat 30), the higher values of probability of crossover p c ,the better results of the system were observed. Similarly,the best mutation rates were relatively low (p m 0:06).We illustrate the average values (out of 10 runs) of thebest values found for dierent probabilities of mutation(with xed p c = 1:0) in Figure 5.f(x)0:0 x i 10:0, i = 1; 2; 3; 4.The global solution is X = (1; 1; 1; 1), and f(X ) = 0.We experimented with two simple systems (A andB); both of them use two operators, and one of these operators(in both systems) was a non-uniform mutation(described in section 2). The only dierence betweensystems A and B was, that the former system used arithmeticalcrossover, i.e., ospring ~z of parents ~x and ~y wasdened asz i = a x i + (1 a) y i for all 1 i n andrandom 0 a 1,whereas the system B used geometrical crossover:z i = x a i y1 ai0 a 1.for all 1 i n and randomFigure 6 illustrates the dierence in the performance ofsystems A and B.1.0f(x)0.50.00.00.060.25P m101−110−210−310−410−510********Figure 5: The performance of the system with p c = 1:0and a variable mutation rate p m01 2 3 4 5 6 7x 10 4Clearly, geometrical crossover can be applied only forproblems where each variable takes nonnegative valuesonly, so its use is quite restricted in comparison withother types of crossovers (e.g., arithmetical crossover).However, for many engineering problems, all problemvariables are positive; moreover, it is always possible toreplace variable x i 2 ha i ; b i i which can take negative values(i.e., where a i < 0) with a new variable y i = x i a i .6 Further Experiments and ResultsWe compared arithmetical and geometrical crossovers onseveral test cases. The preliminary experiments indicatethat geometrical crossover outperforms arithmeticalcrossover on majority of unconstrained test problems.For example, one of the selected test cases was tominimize f(X) = 100(x 2 x 2 1) 2 + (1 x 1 ) 2 +90(x 4 x 2 3) 2 + (1 x 3 ) 2 +10:1((x 2 1) 2 + (x 4 1) 2 )+19:8(x 2 1)(x 4 1),subject to:Figure 6: The performance (number of generations in10,000s versus the average value of the objective functionover 20 runs) of the systems A (marked by `') and B(marked by `*') on one test case7 ConclusionsIt seems that the experiments described in the previoussections can be generalized in the following sense. Initialexperiments (using any evolutionary method) mayindicate active constraints for a given problem. Thenit might be possible to build a specialized system withoperators to search a boundary between feasible and infeasibleparts of the search space. This was precisely thecase of the problem described in section 4; it might bethe case for many other optimization problems.Let us consider, for example, the following optimizationproblem:wheremaximize f(~x) = ( p n) n Q ni=1 x i6

P ni=1 x2 i 1 (6)and 0 x i 1, for 1 i n.Clearly, the objective function reaches its global optimumat~x = h1= p n; : : :; 1= p ni,and f(~x ) = 1.Again, it is relatively easy to initialize the populationby boundary points, i.e., points, satisfyingP ni=1 x2 i = 1.Then, the sphere crossover produces an ospring ~x 3 fromtwo parents ~x 1 and ~x 2 in the following way:x 3 i = p ((x 1 i )2 + (x 2 i )2 )=2.As for geometrical crossover, it is easy to generalize spherecrossover:x 3 i = p (x 1 i )2 + (1 )(x 2 i )2 ,for 0 1, as well as for multiple parents:x k+1i= p 1 (x 1 i )2 + : : : + k (x k i )2 ,for 1 + : : : + k = 1.Clearly, ~x 3 lies also on the sphere dened by (6). Similarly,a problem-specic mutation (for a parent vector ~x)can select two indices, i 6= j and a random number p fromthe range h0; 1i, and assign:= p x t i , andj = q x t j,x t+1ix t+1where q = q ( xixj )2 (1p 2 ) + 1. Such a simple evolutionarysystem nds the global optimum easily (e.g., the caseof n = 20 requires 10,000 generations with populationsize equal to 30, p c = 1:0, and p m = 0:06).It seems that problem-specic operators which searchthe boundary of a feasible region can enhance the searchfor a global optimum in a signicant way. At least forsome numerical optimization problems, it is an interestingoption to explore. Resulting evolutionary systemsare very easy to construct (less than 200 lines of code)and use. As indicated in section 6, such operators canbe useful also for problems without constraints. Weplan further study on properties of various crossoversand boundary-search operators for numerical optimizationproblems.AcknowledgmentsThis material is based upon work supported by the NationalScience Foundation under Grant IRI-9322400.ReferencesBack, T., F. Homeister and H.-P. Schwefel (1991). ASurvey of Evolution Strategies. In Proceedings of theFourth International Conference on Genetic Algorithms,Los Altos, CA, Morgan Kaufmann Publishers, 2{9.Bean, J.C. and A.B. Hadj-Alouane (1992). A Dual GeneticAlgorithm for Bounded Integer Programs. Departmentof Industrial and Operations Engineering, The Universityof Michigan, TR 92-53.Davis, L. (1989). Adapting Operator Probabilities in GeneticAlgorithms. In Proceeding of the 3rd InternationalConference on Genetic Algorithms, Morgan KaufmannPublishers, pp.61{69.Eiben, A.E., P.-E. Raue, and Zs. Ruttkay (1994). GeneticAlgorithms with Multi-parent Recombination. InProceedings of the Third International Conference on ParallelProblem Solving from Nature (PPSN), Springer-Verlag, New York, pp.78{87.Fogel, D.B. (1992). Evolving Articial Intelligence. PhDThesis, University of California, San Diego.Glover, F. (1977). Heuristics for Integer ProgrammingUsing Surrogate Constraints. Decision Sciences, Vol.8,No.1, pp.156{166.Glover, F. and G. Kochenberger (1995). Critical EventTabu Search for Multidimensional Knapsack Problems.In Proceedings of the International Conference on Metaheuristicsfor Optimization, Kluwer Publishing, pp.113{133.Hadj-Alouane, A.B. and J.C. Bean (1992). A GeneticAlgorithm for the Multiple-Choice Integer Program. Departmentof Industrial and Operations Engineering, TheUniversity of Michigan, TR 92-50.Con-Simu-Homaifar, A., S. H.-Y. Lai and X. Qi (1994).strained Optimization via Genetic Algorithms.lation, 62: 242{254.Joines, J.A. and C.R. Houck (1994). On the Use of Non-Stationary Penalty Functions to Solve Nonlinear ConstrainedOptimization Problems With GAs. In Proceedingsof the Evolutionary Computation Conference|PosterSessions, part of the IEEE World Congress on ComputationalIntelligence, Orlando, 26{29 June 1994, 579{584.Keane, A., (1994). Genetic Algorithms Digest, Thursday,May 19, 1994, Volume 8, Issue 16.Kelly, J.P., B.L. Golden, and A.A. Assad (1993). LargeScale Controlled Rounding Using Tabu Search with StrategicOscillation. Annals of Operations Research, Vol.41,pp.69{84.7

Le Riche, R., C. Knopf-Lenoir, and R.T. Haftka (1995).A Segregated Genetic Algorithm for Constrained Optimizationin Structural Mechanics. In Proceedings of theSixth International Conference on Genetic Algorithms,Los Altos, CA, Morgan Kaufmann Publishers, pp.558{565.Michalewicz, Z., (1995). Genetic Algorithms, NumericalOptimization, and Constraints. In Proceeding of the 6thInternational Conference on Genetic Algorithms, MorganKaufmann Publishers, pp.151{158.Michalewicz, Z. and N. Attia (1994). In EvolutionaryOptimization of Constrained Problems. Proceedings ofthe 3rd Annual Conference on Evolutionary Programming,eds. A.V. Sebald and L.J. Fogel, River Edge, NJ,World Scientic Publishing, 98{108.Schwefel, H.-P. (1981). Numerical Optimization for ComputerModels. Chichester, UK, Wiley.Smith, A. and D. Tate (1993). Genetic OptimizationUsing A Penalty Function. In Proceedings of the FifthInternational Conference on Genetic Algorithms, Los Altos,CA, Morgan Kaufmann Publishers, pp.499{503.Wright, A.H. (1991). Genetic Algorithms for Real ParameterOptimization. In Foundations of Genetic Algorithms,ed. G. Rawlins, First Workshop on the Foundationsof Genetic Algorithms and Classier Systems, LosAltos, CA, Morgan Kaufmann Publishers, 205{218.Michalewicz, Z., T.D. Logan and S. Swaminathan (1994).Evolutionary Operators for Continuous Convex ParameterSpaces. In Proceedings of the 3rd Annual Conferenceon Evolutionary Programming, eds. A.V. Sebald andL.J. Fogel, River Edge, NJ, World Scientic Publishing,84{97.Michalewicz, Z. and G. Nazhiyath, G. (1995). GenocopIII: A Co-evolutionary Algorithm for Numerical OptimizationProblems with Nonlinear Constraints. In Proceedingsof the 2nd IEEE International Conference onEvolutionary Computation, Perth, 29 November { 1 December1995.Muhlenbein, H. and H.-M. Voigt (1995). Gene PooolRecombination for the Breeder Genetic Algorithm. InProceedings of the Metaheuristics International Conference,Breckenridge, Colorado, July 22{26, pp.19{25.Powell, D. and M.M. Skolnick (1993). Using Genetic Algorithmsin Engineering Design Optimization with NonlinearConstraints. In Proceedings of the Fifth InternationalConference on Genetic Algorithms, Los Altos, CA,Morgan Kaufmann Publishers, 424{430.Renders, J.-M., and H. Bersini (1994). Hybridizing GeneticAlgorithms with Hill-climbing Methods for GlobalOptimization: Two Possible Ways. In Proceedings of theFirst IEEE ICEC, pp.312{317.Richardson, J.T., M.R. Palmer, G. Liepins and M. Hilliard(1989). Some Guidelines for Genetic Algorithms withPenalty Functions. In Proceedings of the Third InternationalConference on Genetic Algorithms, Los Altos,CA, Morgan Kaufmann Publishers, 191{197.Schoenauer, M., and S. Xanthakis (1993). ConstrainedGA Optimization. In Proceedings of the Fifth InternationalConference on Genetic Algorithms, Los Altos, CA,Morgan Kaufmann Publishers, 573{580.8